Live data from Hacker News

Why Ratings Systems Don't Work

goodfil.ms

51–60 of 165 posts

Re: Why Ratings Systems Don't Work

#51
post #34

Earlier quoted context omitted.

What use case are you referring to? I don't know anyone who uses ratings sites after seeing a movie.

My point exactly. And why would you care about rewatchability if you didn't see the movie once?

Possibly to know whether you should rent it or buy it.

Re: Why Ratings Systems Don't Work

#52
Hmm. "rewatchability" to me is loaded with multiple meanings. That's a problem.

I take rewatchability as possibly meaning complex or rich, in that there is something to be gained by watching it again. For example, Prometheus is rewatchable for me. I can also see it as meaning "a vacuous thrill ride I want to try again". Transformers is rewatchable, but only because it appeals to my childhood daydreams of bads, giant robots. It is certainly not deep.

Second, are there movies in the database high in rewatchability but low in quality? If not, then the variable are essentially dependent and the scatter plot useless.

Re: Why Ratings Systems Don't Work

#53
post #34

Earlier quoted context omitted.

What use case are you referring to? I don't know anyone who uses ratings sites after seeing a movie.

My point exactly. And why would you care about rewatchability if you didn't see the movie once?

rewatchability should indicate a light-weight movie. If you want that, you look at rewatchability, if you want something more complex, you use quality. I think this is the underlying assumption.

Re: Why Ratings Systems Don't Work

#54

I'll say histograms are useful for certain things. If you are looking at electronic devices or camera lenses there's the issue that a certain fraction of people get lemons. Some bad reviews are because of that. Other people have unrealistic expectations of the product and give a bad review. A histogram gives some immediate insight into this problem, and then looking at stratified samples of the reviews helps there on…

Some bad reviews on Amazon are from shipping snafus. If you're going to get any useful info, you have to read the reviews.

Or worse yet, reviews bitching about the price.

Edited to add: "If you're going to get any useful info, you have to read the reviews," is so incredibly true. I'm surprised it's not getting more mentions in these comments. The scatter plot is kind of cool, but I'd so much rather have a histogram and actual reviews to check so I can find out why the product got those ratings.

Re: Why Ratings Systems Don't Work

#55

Earlier quoted context omitted.

My point exactly. And why would you care about rewatchability if you didn't see the movie once?

rewatchability should indicate a light-weight movie. If you want that, you look at rewatchability, if you want something more complex, you use quality. I think this is the underlying assumption.

I don't think this is right. People watch movies like Blade Runner and Citizen Kane repeatedly.

Re: Why Ratings Systems Don't Work

#56
Hmmm. Whatever the scientific or theoretical improvement such an approach may offer having to educate users on how your ratings system works is going to add a huge amount of friction to user engagement.

And frankly who has ever mentally rated a film in terms of "re-watchableness".? People just think in terms of of "good" or "bad" and current ratings systems a la Amazon leverage that. It's simple, fast and given the histogram presentation tells me everything I need to know about the number and distribution of votes in a flash. Plus whether I want to rewatch a film or re-read a book is largely down to my mood at the time. But my opinion on whether it's "good" or "bad" is pretty static.

Maybe Amazon's system is not statistically bullet-proof, but who cares? We're talking movies here: a cheap, casual and discretionary purcahse.

Re: Why Ratings Systems Don't Work

#57
So, watch this if you’re in the mood for something really good.

Something I never see addressed: what you want to watch eventually rarely correlates with what you want to watch now. On the whole, you'll spend lots of time putting "good" movies on your queue, but when time comes to pick something and hit Play you'll pass over those and pick some recent release which engages your excitement now and will be long forgotten soon after.

The star rating system confuses this even more by relying entirely on people who will bother to rate something at all - a very different crowd than the "what's good?" and "thrill me now" mindsets.

Insofar as ratings exist, I focus on written 1-star reviews (movies, apps, products, whatever), looking for a subclass of "there really was a particular problem" comments.

Re: Why Ratings Systems Don't Work

#58

The old Latin proverb "Quis custodiet ipsos custodes?" http://en.wikipedia.org/wiki/Quis_custodiet_ipsos_custodes%3... might in this context be paraphrased to "Who is rating the raters?" The hope in any online rating system is that enough people will come forward to rate something that you care about so that the people who have crazy opinions will be mere outliers among the majority of raters who share your well info…

Many reviewers are biased and judgemental. I prefer to look at the distribution of ratings that amazon shows. Good items have a distribution with one peak, even if it isn't at 5 stars. When ratings create a saddle the product is probably a fluke.

Re: Why Ratings Systems Don't Work

#60

Terrible article. Calling histograms awful, based on nothing more than an opinion. Then trying to conclude that some convoluted scatter plot system makes more sense is laughable. Not to mention, this system is still just a star rating system. This would be no different than having two histograms side by side.. assuming, of course, that you'd even want to rate different aspects of the same thing. I can't even imagine…

I agree, the good idea was to add more than one dimension to the rating. I suspect that it would be even more interesting if we had a couple more ratings to separate that one number into more precise groups.
Post reply on HN