How useful was the Netflix Prize challenge for Netflix? • Hacker News Board

Re: How useful was the Netflix Prize challenge for Netflix?

#131

Post by RandomLensman » Wed, Jul 07, 2021, 8:41 AM UTC

koonsolo wrote: post #92

I don't get why they wanted to go all out on AI for this. I use Netflix, and a simple algorithm would already be an improvement: - If I watched a movie and gave it a thumbs down, don't recommend it - If I watched a movie - If I browsed over a movie 50 times, read the info, and still didn't play it, stop recommending it. - If I watched the last episode, remove the "new episodes" banner. WTF

Yes, I got annoyed, too, by the suggestions on Netflix.

However, what they do might actually work for them, i.e.

(1) The HN is probably not representative of their audience as a whole (2) What they do now might be RoE accretive for them, but not so great even for a wider section of their customers

Could still be suboptimal and probably is...

Re: How useful was the Netflix Prize challenge for Netflix?

#132

Post by Semaphor » Wed, Jul 07, 2021, 8:49 AM UTC

dspillett wrote: post #116

laurent92 wrote: post #101

Earlier quoted context omitted.

I suspect their algorithm is stuck in a local minimum, where it has proven to itself that, if a movie is presented to you hundreds of time, there will be a moment where you click, either inadvertently or because there is nothing else presented to you, and it counts as a validation of their engine. It is so optimized for this that it doesn’t try just presenting all movies anymore - which is a recurrent problem in A/B…

> Yes, Netflix’ engine is the reason I left Netflix That seems rather odd to me. I can believe it was a final straw after other reasons like running out of content you particularly want (absolutely or in comparison with other services), but not it begin "the" reason. I don't particularly pay attention to the recommendations on either Netflix or Amazon, instead picking up things I might like to try from external sourc…

For me, the reason was partially the engine, partially their active work at confusing me.

I would keep seeing stuff I don’t want to watch, and they would keep switching out thumbnails to trick me into thinking it’s something I haven’t encountered. Both of those combined made browsing a chore, and I simply had no interest in using something that’s actively working against me (which is also the reason I went from being an active FB user to only using it for groups and messaging, so it’s not as if "actively working against me" is a Netflix exclusive)

A more minor reason is the lack of information displayed, but I could have handled that.

Re: How useful was the Netflix Prize challenge for Netflix?

#133

Post by codesternews » Wed, Jul 07, 2021, 8:49 AM UTC

koonsolo wrote: post #92

I don't get why they wanted to go all out on AI for this. I use Netflix, and a simple algorithm would already be an improvement: - If I watched a movie and gave it a thumbs down, don't recommend it - If I watched a movie - If I browsed over a movie 50 times, read the info, and still didn't play it, stop recommending it. - If I watched the last episode, remove the "new episodes" banner. WTF

How the heck youtube algo is so good? Just start with incognito tab and once it hints your interest he can put you in Rabit Hole.

I really dislike it. I wanted it to be more like netflix so I don't get suck in.

Re: How useful was the Netflix Prize challenge for Netflix?

#134

Post by dalbasal » Wed, Jul 07, 2021, 8:52 AM UTC

thom wrote: post #88

Do people feel that Netflix has a large enough catalog that its recommendation system really matters? The only useful feature it ever had for me was the ‘new this week’ category that seems to have been retired.

Really matters could be interpreted different ways. Expressed in startup-metric terms, it might be "% of users that watch something" or "time spent browsing before watching something." Any real improvement on such is probably important.

Think of it like Google's famous obsession with speed. Did returning search results 17ms faster really matter? It's hard to say for sure, but I suspect it did.

That said, I agree personally. I don't like Netflix' UI. I suspect you could hand code a browsing/ranking UI of similar value, from a casual users' perspective.

>> large enough catalog

I think this is a case where Netflix didn't end up where they expected the. I think they expected to have a vast catalogue... a "spotify of movies." It just didn't go that way.

You could also reverse the question. Does netflix have a big enough dataset to make a great recommendation system? I think this might be the more pertinent question. Google & FB have their vast ad-centric datasets. I suspect these could be used to make a recommendation engine that's a lot better.

They haven't really done this for youtube though. The priority is to match ads to users. For this, they're willing to push the envelope on how they use user data. For youtube recommendations, it doesn't seem that youtube gets access to much data from outside of youtube.

Re: How useful was the Netflix Prize challenge for Netflix?

#135

Post by michaelt » Wed, Jul 07, 2021, 8:58 AM UTC

fighterpilot wrote: post #117

koonsolo wrote: post #92

I don't get why they wanted to go all out on AI for this. I use Netflix, and a simple algorithm would already be an improvement: - If I watched a movie and gave it a thumbs down, don't recommend it - If I watched a movie - If I browsed over a movie 50 times, read the info, and still didn't play it, stop recommending it. - If I watched the last episode, remove the "new episodes" banner. WTF

The most annoying thing about these streaming platforms is no IMDB rating being displayed. I have to type in every suggestion one by one which is annoying, or use a third party search engine. It'd be such a huge UX boost to simply include the rating, and better yet allow filtering by rating, I really don't get it. Perhaps they're trying to build network effects around their own rating system but it really detracts fr…

> I really don't get it

Presumably poorly rated movies are a lot cheaper to license; and when you're licensing 2/10 rated movies, user satisfaction is a lot higher if you don't show the ratings.

Re: How useful was the Netflix Prize challenge for Netflix?

#136

Post by samplenoise » Wed, Jul 07, 2021, 8:58 AM UTC

The author is still underselling the significance of the progress made during the first years IMO. The simple idea that is still behind most practical recommender systems (using gradient descent to do SVD to complete the rating matrix) was first described in 2006 by Simon Funk [1]. Koren, who ended up taking home a big part of the prize, recently wrote another paper about how that basic idea still outperforms most “AI” (deep neural) recommenders today [2].

[1] https://sifter.org/~simon/journal/20061211.html

[2] https://arxiv.org/abs/1905.01395

Re: How useful was the Netflix Prize challenge for Netflix?

#137

Post by SiempreViernes » Wed, Jul 07, 2021, 9:01 AM UTC

fighterpilot wrote: post #117

koonsolo wrote: post #92

I don't get why they wanted to go all out on AI for this. I use Netflix, and a simple algorithm would already be an improvement: - If I watched a movie and gave it a thumbs down, don't recommend it - If I watched a movie - If I browsed over a movie 50 times, read the info, and still didn't play it, stop recommending it. - If I watched the last episode, remove the "new episodes" banner. WTF

The most annoying thing about these streaming platforms is no IMDB rating being displayed. I have to type in every suggestion one by one which is annoying, or use a third party search engine. It'd be such a huge UX boost to simply include the rating, and better yet allow filtering by rating, I really don't get it. Perhaps they're trying to build network effects around their own rating system but it really detracts fr…

Maybe it's because filtering by rating would expose how shallow the catalogue is?

At least in my locale a good way to be reminded of this is searching for any movie you'd like to watch but isn't on the front page of netflix, typically they won't have it.

Re: How useful was the Netflix Prize challenge for Netflix?

#138

Post by eythian » Wed, Jul 07, 2021, 9:03 AM UTC

birdyrooster wrote: post #112

manigandham wrote: post #108

Earlier quoted context omitted.

There’s not enough content for completely new listings every time, and people do watch things repeatedly, especially TV shows.

I get the value of rewatching stuff, but some people take it to an extreme and I suspect these were the same people whose parents let them watch the same children’s show/movie over and over and over.

My understanding is that children rewatching things over and over is healthy for them, as it aids in things like language learning.

Anecdotally, in my own experience learning a second language, the online part of the course will play an audio or video clip, then play it again and have you answer questions on it, then do it again having you fill out missing words in a transcript, then play it again as you read the text along. I've found this extremely helpful to tune my ears.

Re: How useful was the Netflix Prize challenge for Netflix?

#139

Post by SiempreViernes » Wed, Jul 07, 2021, 9:05 AM UTC

matkoniecz wrote: post #118

dspillett wrote: post #116

Earlier quoted context omitted.

> Yes, Netflix’ engine is the reason I left Netflix That seems rather odd to me. I can believe it was a final straw after other reasons like running out of content you particularly want (absolutely or in comparison with other services), but not it begin "the" reason. I don't particularly pay attention to the recommendations on either Netflix or Amazon, instead picking up things I might like to try from external sourc…

I cancelled Netflix subscription because I was not finding anything worth watching. I tried downvoting unwanted featured movies, they continued to be featured. Once my listing was filled solely with already watched or downvoted movies I cancelled account. I suspect that Netflix had some movies that I would gladly watch, but if they show what I explicitly marked as invented...

> I suspect that Netflix had some movies that I would gladly watch

No so sure about that, in my experience they really don't have a deep licence pool any more, at least not for movies.

Re: How useful was the Netflix Prize challenge for Netflix?

#140

Post by bryanrasmussen » Wed, Jul 07, 2021, 9:05 AM UTC

TeMPOraL wrote: post #126

bryanrasmussen wrote: post #121

Earlier quoted context omitted.

>- If I watched a movie yeah some movies might be more likely to be rewatchable quicker by a larger amount of the population, and some people might be more likely to be able to rewatch movies they like more quickly than a month - and those people might have preferences that indicate their liking to rewatch more often. Thus it might be nice to do AI on this.

All the AI you need for this is an input box: Wait [ ] months before recommending again a show I already watched. Alas, giving users control is anathema to this industry.

as I said >yeah some movies might be more likely to be rewatchable quicker by a larger amount of the population,

>Wait [ ] months before recommending again a show I already watched.

does not handle that scenario, furthermore I am perhaps more pessimistic about user self knowledge than you are. I would probably make that box some high number but I am a sucker for rewatching some movies very often.