Live data from Hacker News

Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

variety.com

61–70 of 298 posts

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#61

> The streaming service said it had been testing thumbs up and down ratings “with hundreds of thousands of members” in 2016 – and it led to 200% more ratings being given. I don't understand why engagement is the right metric here. If someone isn't sure how they feel about a movie, why is it a benefit to have them spew their half-formed thoughts into a like/dislike rating?

Here's the benefit - getting a clearer picture of your tastes, which allows them to make you better recommendations. Their recommendation engine would be based on finding users similar to you and telling you what they liked. More ratings is better than nuanced ratings for this engine because you're more likely to find overlap between users.

The goal isn't making a database of the "best" and "worst" movies and TV shows. We already have the International Movie Database for that. This is just about finding out what you like, and finding other stuff like it.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#62
I think a lot of people are reacting here as if Netflix is moving from the only signal they have being the star and review system to the only signal they have being the up/down system, and thus confused as to why Netflix would throw away the vast majority of their rating ability.

But that's not even close to true. In addition to it technically being up/down/no rating, we've got how long people watch the show for, how the shows they watch form a pattern, how all the patterns of what people watch show global preference patterns, whether they rewatch a show, when they watch what sort of shows, what individual scenes are rewatched vs. skipped... Netflix is swimming in a sea of preference data, not sitting here trying to figure out "Gosh, um, if the user likes this movie 3 vs. 4 stars, uh... what does that mean?"

It makes perfect sense to me to optimize this one-data-stream-among-many to increase user participation and get more bits of information from more people engaging with the simpler system, rather than trying to squeeze bits out of the few people willing to use the star system and the even fewer willing to write useful reviews.

It isn't really even as shocking as it may seem at first. The star system has 6 states, "no rating", 1, 2, 3, 4, and 5 stars. That's 2.6 bits, with some simplifying assumptions [1]. The thumb system has up, down, and no rating; that's 1.6 bits [1]. To make up for the bits, you need only see ~40% increase in participation over the current star system... think that's going to happen?

[1]: The simplifying assumption is that all outcomes are equally likely, but that's not true. I don't have the numbers to run a more complete information theory analysis, but it's not hard to imagine the "no star rating" case is so common that it produces such a small fractional bit that a higher-participation-rate yea/nay/no rating (if you puth this UI in their face, "no rating" becomes much more meaningful, too) straight-up produces more bits of information on average, and is thus simply an improvement even before considering the superior UI experience. I rather suspect this is the case, some very sensible assumptions would suggest this, but I lack the ability to prove this; the "assume all outcomes are equally likely" is at least a concrete case I can discuss.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#63

If you have many reviewers, you don't need to get more information than thumbs up or down from each person. How much people thumbed it up is a good indicator. Star-rating can be too much detail, anyway: if you're comparing two shows, and one has a higher fraction of five-star ratings, but the other has a higher fraction of ratings that are four or above, which is better? Star ratings can be too much detail and cause…

I do find IMDB's ratings breakdown fascinating to explore, though.

For example, you can look at the 50 Shades films or A Dog's Purpose or (especially) Gunday and have a pretty good idea that lots people are giving them bad ratings without having seen the films. (Their weighted average ratings are supposed to combat that, but they don't seem to be doing a very good job.)

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#64
post #42

I wish there was a "meh" option. Sometimes things are ok, but not good or bad. This will leave me not rating a bunch of things unless they're good or bad.

I'd bet Netflix uses the watched but not rated signal, especially if you often rate things.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#65

If you have many reviewers, you don't need to get more information than thumbs up or down from each person. How much people thumbed it up is a good indicator. Star-rating can be too much detail, anyway: if you're comparing two shows, and one has a higher fraction of five-star ratings, but the other has a higher fraction of ratings that are four or above, which is better? Star ratings can be too much detail and cause…

The goal isn't to broadly determine which movies are good and bad; the goal is to make accurate predictions about which titles a particular person will enjoy.

One good way to accomplish this is to pool users into cohorts of similar tastes, then suggest movies enjoyed by their cohort but not yet watched by a given user.

The question is whether doubling the frequency of rating a title is more valuable than a slightly more nuanced understanding of what each person likes. 'More valuable' meaning they can build you a better cohort from which to make title suggestions.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#66
Terrible idea, thumbs up or down rating only promote political reviews, a movie that makes critics of feminism? thumbs down despite the acting or contents inmediatly, a movie showing the good parts of religion? thumbs down by viceral atheists, some one just needs to look at youtube and see how this doesn't work,

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#67
post #41
post #2

And not only that, but also removing their existing collections of videos and replacing it with their "Made by Netflix" shows that mostly amount to shovelware. I'm about ready to drop them, just like they've dropped MASH, soon to be X-Files, and many many movies.

Sigh. Why do so many not seem to understand that, in most cases, Netflix is likely not "dropping" anything? Netflix does not own the rights to the content and the owners can decide on the terms of the agreement on how long the content will be on Netflix. People threatening to leave, or actually leaving, lowers Netflix's leverage to try to get and keep the content in their system. This is why Netflix is creating their…

Sure, that makes sense, but as a consumer, all that matters to me is whether they have the content I want or not. If they don't, Amazon will gladly sell me the season DVD's that Netflix hasn't been able to host. I'm not personally interested in 'investing' in Netflix creating their own content (which I generally haven't been that into honestly).

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#68
post #57

Earlier quoted context omitted.

>> Star ratings can be too much detail and cause confusion The iOS App Store is a good example of this. An unbelievable number of people, for some reason, thing 1 star = great and 5 star = bad.

Haha yes! Where the hell do these people learn that?? The other thing that annoys me with ratings in general, specifically places like amazon, is when people pan a perfectly good item because of some third party factor. "Item was great, but it came in a beat up box from the post office, I'm giving it one star." WTH does someone drop kicking your box have to do with the product?

I have to think that some of those reviews are just trolls.

Bad reviewers will always be part of the system. You have to expect it. Netflix has tons of reviews that go "I turned this off before the opening credit sequence finished, worst movie ever!!!! -- 1 star".

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#69
post #57

Earlier quoted context omitted.

>> Star ratings can be too much detail and cause confusion The iOS App Store is a good example of this. An unbelievable number of people, for some reason, thing 1 star = great and 5 star = bad.

Haha yes! Where the hell do these people learn that?? The other thing that annoys me with ratings in general, specifically places like amazon, is when people pan a perfectly good item because of some third party factor. "Item was great, but it came in a beat up box from the post office, I'm giving it one star." WTH does someone drop kicking your box have to do with the product?

The other thing you see in ratings: "I've patronized this business every day for 5 years and always loved it. Then the service was kinda rude the other day. 1 star. Can't believe they stay in business."

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#70
I feel like star ratings, if just given in general, are pretty useless for anything that has multiple qualities. For example, maybe I like the cinematography of a movie but I think the acting is shit, or maybe I think the taste of the beer is nice but it's too acidic or sour and wouldn't work with specific foods because it's overpowering.

Airbnb does this pretty much spot on, in that they ask for 1-5 stars for five categories, rather than just "did you enjoy this Airbnb?" I also think that Untappd does this wrong, because in general beers can't be described in just a 1-5 star rating without deeper explanation.

I think that moving from a pure star rating to a thumbs up and down rating is better overall, if only because it makes me, as a watcher, not have to think as much and therefore give a rating where I might not have before. If I want to go more in depth, I can explain more too.

Post reply on HN