How useful was the Netflix Prize challenge for Netflix? • Hacker News Board

Re: How useful was the Netflix Prize challenge for Netflix?

#121

Post by bryanrasmussen » Wed, Jul 07, 2021, 8:08 AM UTC

koonsolo wrote: post #92

I don't get why they wanted to go all out on AI for this. I use Netflix, and a simple algorithm would already be an improvement: - If I watched a movie and gave it a thumbs down, don't recommend it - If I watched a movie - If I browsed over a movie 50 times, read the info, and still didn't play it, stop recommending it. - If I watched the last episode, remove the "new episodes" banner. WTF

>- If I watched a movie yeah some movies might be more likely to be rewatchable quicker by a larger amount of the population, and some people might be more likely to be able to rewatch movies they like more quickly than a month - and those people might have preferences that indicate their liking to rewatch more often.

Thus it might be nice to do AI on this.

Re: How useful was the Netflix Prize challenge for Netflix?

#122

Post by studentrob » Wed, Jul 07, 2021, 8:10 AM UTC

Oh man this was huge. You have to remember, machine learning was nothing back then. It was on nobody's radar.

Then comes this flashy $1 million prize. Tons of universities had teams. So it really helped their recruitment.

It also likely contributed to the idea of creating Kaggle which has itself greatly contributed to data-science education by giving everyone an open forum in which to compete.

Then there were other signficant projects around this time like ImageNet which became a competition too. That open dataset led to tons of research and applications.

Re: How useful was the Netflix Prize challenge for Netflix?

#123

Post by bryanrasmussen » Wed, Jul 07, 2021, 8:10 AM UTC

klausjensen wrote: post #97

koonsolo wrote: post #92

I don't get why they wanted to go all out on AI for this. I use Netflix, and a simple algorithm would already be an improvement: - If I watched a movie and gave it a thumbs down, don't recommend it - If I watched a movie - If I browsed over a movie 50 times, read the info, and still didn't play it, stop recommending it. - If I watched the last episode, remove the "new episodes" banner. WTF

Completely agree. I am constantly baffled at the bullshit Netflix suggests me. I just went to the frontpage of my Netflix and...: - "My list" recommends 3 series where I have alread watched the last episode - "Only on Netflix" recommends another two series, I have already wathed. - A section displayed is "Watch together for older kids" - I dont have kids, never watched any kids stuff on my account. - "Documentaries"…

>- "Only on Netflix" recommends another two series, I have already wathed.

I think the only on Netflix part works as part of positive reinforcement - it always shows me stuff it knows I watched all the way through (hence I must have liked it) mixed with things I haven't watched or haven't watched in a long time.

Thus when I see that section it is reminding me they have stuff I liked that I can only get there so please don't ever change account, and here is some more of that stuff only we got - try it out!

Re: How useful was the Netflix Prize challenge for Netflix?

#124

Post by computerphage » Wed, Jul 07, 2021, 8:28 AM UTC

fighterpilot wrote: post #117

koonsolo wrote: post #92

I don't get why they wanted to go all out on AI for this. I use Netflix, and a simple algorithm would already be an improvement: - If I watched a movie and gave it a thumbs down, don't recommend it - If I watched a movie - If I browsed over a movie 50 times, read the info, and still didn't play it, stop recommending it. - If I watched the last episode, remove the "new episodes" banner. WTF

The most annoying thing about these streaming platforms is no IMDB rating being displayed. I have to type in every suggestion one by one which is annoying, or use a third party search engine. It'd be such a huge UX boost to simply include the rating, and better yet allow filtering by rating, I really don't get it. Perhaps they're trying to build network effects around their own rating system but it really detracts fr…

IMDb is owned by Amazon. Amazon Prime shows IMDb ratings in its app.

Re: How useful was the Netflix Prize challenge for Netflix?

#125

Post by TeMPOraL » Wed, Jul 07, 2021, 8:31 AM UTC

szszrk wrote: post #115

dooferlad wrote: post #110

Earlier quoted context omitted.

What Netflix wants to be popular also tramples over what I am already watching. I find that the UI gets in the way of enjoying several programs that I am part way through by pushing something new. If they want to continue to make series and only get me to watch half of the episodes then the algorithm is spot on. That said, Amazon Prime video is so much worse. It suggests that I would like to watch series 1 of a show…

To their defense - their catalog in some countries (like mine) is so small that I was able to just ... browse through the whole offer in 10 minutes. I've managed to binge-watch everything I liked in two months (while working full time remotely) and cancel Prime. With such a small catalog suggestions will mostly be wrong, as there isn't enough content to fill the whole suggestions tab :)

Which makes you wonder why they even bother with suggestions in the first place, instead of doing the obvious thing: giving you a searchable list of all movies they have for your region.

Given that the former is a major engineering project, while the latter is a junior-level interview question, one has to assume they're trying to confuse their users on purpose.

Re: How useful was the Netflix Prize challenge for Netflix?

#126

Post by TeMPOraL » Wed, Jul 07, 2021, 8:34 AM UTC

bryanrasmussen wrote: post #121

koonsolo wrote: post #92

I don't get why they wanted to go all out on AI for this. I use Netflix, and a simple algorithm would already be an improvement: - If I watched a movie and gave it a thumbs down, don't recommend it - If I watched a movie - If I browsed over a movie 50 times, read the info, and still didn't play it, stop recommending it. - If I watched the last episode, remove the "new episodes" banner. WTF

>- If I watched a movie yeah some movies might be more likely to be rewatchable quicker by a larger amount of the population, and some people might be more likely to be able to rewatch movies they like more quickly than a month - and those people might have preferences that indicate their liking to rewatch more often. Thus it might be nice to do AI on this.

All the AI you need for this is an input box:

  Wait [  ] months before recommending again a show
  I already watched.

Alas, giving users control is anathema to this industry.

Re: How useful was the Netflix Prize challenge for Netflix?

#127

Post by dalbasal » Wed, Jul 07, 2021, 8:35 AM UTC

"ROI" is probably the wrong frame for this question. "Worth it" is better. "Better than X" might be better still, since it frames the question such that "X" needs to be defined and quantified.

The benefits of such a competition are pretty nebulous, and there's no way to convince an ardent skeptic. OTOH, many business decisions are like this and skepticism isn't a viable frame in many cases.

Netflix got visibility with investors and potential employees. Netflix's recommendation engine became famous, even though it doesn't seem impressive as a user. The exercise created a structured way of thinking about their recommendation algorithm. They cemented its importance. Even though they didn't implement the winning solution, they did get a useful benchmark. This was potentially very useful in further decisions in R&Ding the recommendation engine in-house.

All that for $1m?

Re: How useful was the Netflix Prize challenge for Netflix?

#128

Post by rocqua » Wed, Jul 07, 2021, 8:36 AM UTC

I used the same dataset for a small project in my CS master. It was a really fun challenge, and it taught me a lot.

Most notably, it taught me that it was incredibly hard to make significant progress past the most simplest and naive approach. That approach was "Take average rating a user gives, take the average rating a movie gets, multiply". (Ratings normalized to be between 0 and 1).

Just using this method would give us 95% of the accuracy of our final method. I think I calculated, and compared to the prize winning result, our method got ~90% as accurate a result.

Re: How useful was the Netflix Prize challenge for Netflix?

#129

Post by rocqua » Wed, Jul 07, 2021, 8:40 AM UTC

IncRnd wrote: post #29

I was a solver on a different team for the netflix challenge. Our team didn't win the grand prize. I would have expected that a blog post would discuss how this was structured. Netflix contracted with innocentive.com, which is a website for solvers, and contracting to that website expanded Netflix's reach to a greater available pool of solvers. As far as I recall, all the allowed solvers for the netflix challenge _ha…

Note, not just the original listing has been taking down, but also the data used in the challenge. I believe because a few people in that set actually got de-anonimized. But certainly because of privacy concerns.

I think the data is still kicking around somewhere in torrent land. I also still have my own copy somewhere, I think.

Re: How useful was the Netflix Prize challenge for Netflix?

#130

Post by sytelus » Wed, Jul 07, 2021, 8:40 AM UTC

Recommandation algorithms are massive sales driver if you depend on long tail. This was true for back in the days and long term subscriber retention would almost entirely depend on great recommandations. Unfortuately, Netflix figured out that they need not depend on long tail to retain their market value. Instead the new idea is same as HBO model. Now the recommandations are basically garbedge.