Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

variety.com

Post reply on HN
Print view View original

131–140 of 298 posts

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#131

Post by shdon »

jerf wrote: post #62

I think a lot of people are reacting here as if Netflix is moving from the only signal they have being the star and review system to the only signal they have being the up/down system, and thus confused as to why Netflix would throw away the vast majority of their rating ability. But that's not even close to true. In addition to it technically being up/down/no rating, we've got how long people watch the show for, how…

> To make up for the bits, you need only see ~40% increase in participation over the current star system... think that's going to happen?

For some people, such as myself, it will decrease participation (making the "no rating" indistinguishable from "it was okay, but I don't love it and don't hate it, so I wouldn't know whether to upvote or downvote"). A 40% increase is also quite a significant change. I'd be astonished if they got that much more engagement out of it.

There's also quite a difference between having the data for their internal algorithms and what people want to see. Just like Facebook eventually added alternatives for just the "Like" button.

I don't mind having a percentage displayed rather than a star rating (I automatically convert everything in my mind anyway, even though the star ratings are more easily visible at a glance when scrolling through a list), but I mourn losing a lot of expressiveness in the ability to record what I really thought of the video that I watched.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#132

Post by wj »

I'm still disappointed they removed their friends feature. I valued my friends' ratings more than the ratings of the Netflix user base as a whole. I have noticed that Netflix tends to be more accurate with the "users like you" star rating than the whole user base ratings.

I no longer see a sum of my ratings but I believe it is well over 2,500. I know others that have rated a lot more (when there was the friends feature you could see your friends' number of ratings). Because of my number of ratings I felt that Netflix did a pretty good job with recommending me content. I will be sad to see this go as I definitely refer to star ratings when adding content to my queue.

Others have mentioned that the two factors of quality and enjoyment make the star rating more valuable and I agree. The only time I remember it breaking down for me was the film Rachel Getting Married (though I am sure there were others). I couldn't stand Anne Hathaway's character to the point that I gave it one star but at the same time I recognized that she gave a really strong performance in what was probably a good film.

Are they converting ratings to thumbs up/down? What does a three-star rating convert to? Those are typically movies that I enjoyed but wouldn't rewatch or recommend to others.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#133

Post by adventured »

ProAm wrote: post #86
pizzetta wrote: post #74

As I posted in the dupe: This saddens me. >Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries That's not incongruous to me. The stars are not about "enjoyment" factor, they are about perceived quality. I may have a go to cheap ice-cream and rate it 3-stars but rate a good affogato 5-stars and only have it once in…

> They are diluting the meaning of quality and instead are opting for a saccharine "enjoyment" factor. They are just trying to increase number of views for a poor quality library. This will mask how bad most of the content is and increase their numbers.

Like many people I've had a Netflix account for a long time - I can't recall the last thing I watched on their streaming service. It has probably been four months since I've used it to watch a movie for example. Their library is increasingly complete trash outside of a few original content programs. I'd cancel it if it weren't so cheap, mostly I keep the subscription as an entertainment fall back (in the category of: well, maybe I'll watch something on Netflix, oh, nope). I suspect I'll just ditch Netflix in the near future and be done with it, as I'm pretty happy with Prime as essentially a freebie to my Amazon shopping.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#134

Post by bryanlarsen »

pizzetta wrote: post #74

As I posted in the dupe: This saddens me. >Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries That's not incongruous to me. The stars are not about "enjoyment" factor, they are about perceived quality. I may have a go to cheap ice-cream and rate it 3-stars but rate a good affogato 5-stars and only have it once in…

They're optimizing the wrong metric.

They should be optimizing "number of people who pay for Netflix". Instead they are optimizing "number of hours spent watching Netflix".

Those two metrics are correlated, but definitely not fully.

If I spend hundreds of hours a month watching meaningless action movies and TV on Netflix, I can cancel my subscription and save myself hundreds of hours and $10. I've got lots of other ways of filling my time with endless pablum.

If I spend 2 hours watching a thought-provoking documentary, I say to myself "yup, that justified spending $10 on Netflix this month."

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#135

Post by Houshalter »

masklinn wrote: post #116
yladiz wrote: post #70

I feel like star ratings, if just given in general, are pretty useless for anything that has multiple qualities. For example, maybe I like the cinematography of a movie but I think the acting is shit, or maybe I think the taste of the beer is nice but it's too acidic or sour and wouldn't work with specific foods because it's overpowering. Airbnb does this pretty much spot on, in that they ask for 1-5 stars for five c…

Incidentally, Steam ratings are also simple up/down, with total and sliding window 30 days aggregate, and an Amazon-style helpful/unhelpful/funny meta-rating.

Steam ratings have their own problems. Like the creators changing something minor and getting thousands of protest negative reviews from upset people. Which doesn't reflect the quality of the game at all. But does discourage new people from trying the game.

My favorite example is Fallout 4. There are tons of negative reviews from people who put 600+ hours into the game. How the hell do you put 25 days of your life into a game you don't like? They claim it "got boring". Not many games will not be boring after 600 hours...

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#136

Post by bluntfang »

bmelton wrote: post #93
bluntfang wrote: post #83

Earlier quoted context omitted.

When I watch netflix, I use the stars as a measure of if I would enjoy something. I don't think most people use stars as a measure of "quality" because when they browse movies, they are looking for something they enjoy, not something that's high quality that they might not enjoy.

This seems like pure difficulty for me. When I think of thumbs up / down metrics I think in terms of "would I endorse this?" A movie like "Suicide Squad" was (to me at least) highly enjoyable, though I thought it objectively had many problems with plot and delivery. I would probably give it 3 stars, which is relatively neutral for me (or leave it altogether unrated). If it were thumbs up/down, I would probably have t…

I think you are an anomalous user. I'm not sure why you're OK with rating 1-5 stars but not a thumbsup or down. I think you're just upset because they are changing.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#137

Post by refrigerator »

DaveWalk wrote: post #92

Question to the Machine Learning folk: are five-star ratings "better" than thumbs up/down, or is it just a matter of algorithm design? I know ratings/prediction has long been studied by the MovieLens.org scientists.

I think the fact that Netflix saw a 200% increase in "thumbs" ratings vs "stars" in the A/B test makes it much better from a machine learning perspective, even if it might at first seem like the data is of "lower quality". One of the biggest problems for recommender systems is that the data is extremely sparse - most users will have only rated a tiny proportion of films, and most films will only have been rated by a tiny proportion of users.

This is just my opinion now, but having studied recommender systems in a decent amount of depth, I don't think the design of the algorithm will need to change. The current techniques that provide the best results use matrix factorisation to simultaneously learn characteristics of films and how much each user likes each characteristic. My intuition would be that the algorithm can learn a lot more about a user from 3 up/down-ratings than from 1 star-rating, and the only reason Netflix are doing this is so that they can provide better recommendations so it's almost certainly the case.

TL;DR: All else being equal, 5-star ratings carry more signal for any machine learning algorithm, but the fact that thumbs up/down will result in 200% more data is a lot more significant than the delta in the signal.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#139

Post by pizzetta »

Fifer82 wrote: post #126
pizzetta wrote: post #106

Earlier quoted context omitted.

When I rate things it's usually an "overall" rating, rather than weigh one dimension disproportionately. Which is why I would almost never grant a 5-star rating to something. But I'll dish out 2, 3 and some 4s regularly.

That is only you, I can't use that information. Like you might be taking into consideration "Special effects were 3 on their own". Whereas my Mum would never even consider that as a thing which influences the rating.

I understand that. It's not perfect and I don't think the assumption is that it was but more that overall people as a group will smooth those rough edges. In other words the majority will compensate for the minority who use the ratings system differently.

For all the faults of the old system, the new one is even worse. Like Pandora, I'll probably just never rate things or mostly just give them a down vote if they don't crack a threshold of goodness.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#140

Post by minimaxir »

It's about damn time.

5-star rating systems are broken. A 1/2/3-star rating is effectively a dislike, and a 4/5-rating is effectively a like. In the big data sense, a deviation from a 1-star for dislike and 5-star for like is statistically meaningless. (and this is universal; star ratings on Amazon Products and Yelp Locations have the exact same distribution).

The interesting part of this is what Netflix gains from the change, since their recommendation algorithms will become less granular. Maybe they came to the same conclusion?

Post reply on HN