Live data from Hacker News

Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

variety.com

101–110 of 298 posts

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#101

Question to the Machine Learning folk: are five-star ratings "better" than thumbs up/down, or is it just a matter of algorithm design? I know ratings/prediction has long been studied by the MovieLens.org scientists.

The technical term for a multi-perference rating is a Likert Scale. There are different schools on this -- an even or odd number of options (even "forces" reviewers to indicate some preference positive or negative, odd allows fence-sitting), how to compare different raters' preferences, etc.

I'm finding interesting the rating system I've created and am trying to apply with some consistency to my Pocket archive of a few thousand items. Nominally it runs from 0 to 5, though I may reserve a 6 for an absolutely mind-blowing piece.

A 0 is a net negative: you are less informed for having read it, it reduces teh intelligence of its reader.

A 1 is, generally, a simple noting of some event.

A 2 should be a general news story, without strong insight.

A 3 is a news or general interest story with strong insight, or a typical scientific paper, or an undistinguished book (generally nonfiction).

A 4 is a particularly good scientific paper, or a typically well-thought-out book.

A 5 is a document which establishes a fundamental idea or field. Claude Shannon's original paper on information theory, say.

I don't think I've run across a 6 yet, but that might be a work which ties together two or more previously unrelated fields into a common theory.

My problem has been in assigning far too many '3' class articles. I've already carved out a list to re-assess and downgrade if appropriate.

I'd also like to be able to report on the numbers for each classification, though Pocket's utter lack of quantitative reporting (I cannot even state how many articles I've collected in total) stymies this.

I am ... increasingly dissastisfied with Pocket as an information management tool.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#102
You could come up with an algorithm that instead of showing an average of 5 stars, biases the weighting towards people who rate things in a similar way to you. We all have an internal set of assumptions about what a 5 star rating system means to us; this is a case where the filter bubble leads to better understanding. People would then see the ratings they expected rather than a mix of different methodologies averaged?

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#103
post #45

Is this just part of the trend to dumb everything down to a polarising dichotomy? Because the trend to stop people from having a more complex and rational opinion than just picking sides doesn't seem to exactly improve discourse or debate, or inform and educate... Wouldn't it be more beneficial to have fewer people give a more detailed review to describe what they like and dislike about something? What use is "good"…

I think one problem is that people have vastly different opinions on what 3 stars mean. For some it's quite bad, for some it's in the middle.

A binary option actually has three options. Did it prompt me to rate it as bad, or did it prompt me to rate it as good, or was it bland enough where I don't want to sway the rating of it at all?

If I rate a movie on IMDB as 5-6, and maybe even 7, the possibility of me recommending it to someone "depends". In that case I might as well not rate it at all.

Ratings are inherently tricky, because we all like different things. I don't think you lose anything by going to a like/dislike system. Recommendations should be based on aggregate maps and matching you with similar viewers that have a certain overlap in ratings. That's made easier with binary choices.

IMDB ratings are only partially informative to me anyways.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#104

As I posted in the dupe: This saddens me. >Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries That's not incongruous to me. The stars are not about "enjoyment" factor, they are about perceived quality. I may have a go to cheap ice-cream and rate it 3-stars but rate a good affogato 5-stars and only have it once in…

> The stars are not about "enjoyment" factor, they are about perceived quality I ignore ratings because of that. They don't mean anything and I am guilty. On Good Reads, I have given what is deemed as a trashy piece of writing 5 stars. At the same time, I have given Dune 1 star because the story was garbage. So what is this 5 star rating? Some people say it is the experience, some say it is the writing style, others…

Your thesis is basically, ratings mean absolutely nothing, and how good something is should solely be based on how popular it is?

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#105
I don't have particularly strong opinions about this change, I don't actually rate many things that I watch (I think the recommendation system can go mostly by what I watch).

I think the fundamental problem with different rating systems is that they are all one-dimensional. Maybe a two-category system, like 1-5 stars for "enjoyment" and another 1-5 stars for "quality". Or a 1-5 rating paired with a label, like "campy" or "serious", to prevent apples-and-oranges comparisons.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#106

As I posted in the dupe: This saddens me. >Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries That's not incongruous to me. The stars are not about "enjoyment" factor, they are about perceived quality. I may have a go to cheap ice-cream and rate it 3-stars but rate a good affogato 5-stars and only have it once in…

> The stars are not about "enjoyment" factor, they are about perceived quality I ignore ratings because of that. They don't mean anything and I am guilty. On Good Reads, I have given what is deemed as a trashy piece of writing 5 stars. At the same time, I have given Dune 1 star because the story was garbage. So what is this 5 star rating? Some people say it is the experience, some say it is the writing style, others…

When I rate things it's usually an "overall" rating, rather than weigh one dimension disproportionately. Which is why I would almost never grant a 5-star rating to something. But I'll dish out 2, 3 and some 4s regularly.

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#107
post #34
post #26

What I really want to avoid is the mediocre content i.e. 2.0 - 3.5 stars, very good and very bad can both be entertaining.. Now that mediocre content will be scattered across the two categories I really care about; I guess IMDB/Rotten tomatoes will steer clear of this ridiculous binary rating systems.

Isn't Rotten Tomatoes exactly a (aggregated) binary rating system?

Yes it is, and it sometime gives some strange ratings like "Ash vs Evil: season 2" having a 100% rating[0]. Though it's probably a good show, it's not absolute perfection like the rating would imply.

0: https://www.rottentomatoes.com/tv/ash_vs_evil_dead/s02

Re: Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

#109

Earlier quoted context omitted.

> The stars are not about "enjoyment" factor, they are about perceived quality I ignore ratings because of that. They don't mean anything and I am guilty. On Good Reads, I have given what is deemed as a trashy piece of writing 5 stars. At the same time, I have given Dune 1 star because the story was garbage. So what is this 5 star rating? Some people say it is the experience, some say it is the writing style, others…

Your thesis is basically, ratings mean absolutely nothing, and how good something is should solely be based on how popular it is?

The problem as I understand it is that the "rating space" is multidimensional (a piece can be funny with great plot but bad writing compensated for by a great lead and awesome special effects although the set design was inconsistent) but all of this has to be projected into a 1-dimensional system.

And the rub is that not everybody uses the same projection, which means the utility of one person's rating for another unknown their projection system (the weight they give to each sub-component of the overall rating) is completely unknown.

Post reply on HN