Live data from Hacker News

Why Ratings Systems Don't Work

goodfil.ms

21–30 of 165 posts

Re: Why Ratings Systems Don't Work

#21

Terrible article. Calling histograms awful, based on nothing more than an opinion. Then trying to conclude that some convoluted scatter plot system makes more sense is laughable. Not to mention, this system is still just a star rating system. This would be no different than having two histograms side by side.. assuming, of course, that you'd even want to rate different aspects of the same thing. I can't even imagine…

Possibly correct but a bit grumpily put.

Re: Why Ratings Systems Don't Work

#23
post #17

The article mentions an important point about 5 star systems - people tend to only use 5 stars or 1 star. This is sort of shown with the histograms. A 2 axis system seems like a good idea. But I'd like to see it with 3 options per access - [UP] [INDIFFERENT] [DOWN]. I'm also interested to know how the system will cope with "controversial" films ( life of brian , for example) where some people are going to downvote wh…

I'd say people only use "5" (really liked, would recommend) and "meh" (spent some time, would not recommend)

In their example, blade runner has sligtly more "meh" votes and starship troopers is mostly "meh"

Re: Why Ratings Systems Don't Work

#24
Why would someone who is looking for a film care about rewatchability? Presumably they'll be watching it for the first time and then can decide for themselves whether to ever watch it again.

Or did I miss the point and rewatchability is just a placeholder for something more useful?

Re: Why Ratings Systems Don't Work

#25
Actually, I find histograms pretty useful, primarily because if there is a secondary bump toward 1, it indicates there are a significant number of people who had bad experiences with the product -- more investigation required.

Having a two-dimensional graph might have more information, if the dimensions really matter. I'm doubtful that "stars" and "rewatchable" are really independent, and I'm unsure why I would care about it when I haven't seen the film. (If I have seen the film, I'll have my own opinion and not need the graph.)

I'm all for looking for improvements to the ratings game, though. What seems to work best for me is to actually read the reviews, but that's obviously time-intensive.

Re: Why Ratings Systems Don't Work

#26
Often I'm presented with a choice of films at the cinema (some of which I've seen already) and only want to decide which one to see next. Perhaps an ordering by score would be more appropriate?

Re: Why Ratings Systems Don't Work

#27
post #8

Terrible article. Calling histograms awful, based on nothing more than an opinion. Then trying to conclude that some convoluted scatter plot system makes more sense is laughable. Not to mention, this system is still just a star rating system. This would be no different than having two histograms side by side.. assuming, of course, that you'd even want to rate different aspects of the same thing. I can't even imagine…

This would be no different than having two histograms side by side.. Yes it would, and the article shows why and how. Scatter plots are easy to read (for comp./math. educated people). Two histograms side by side are easy to read (and find correlations) for nobody.

The point is that scatter plots are harder to read than histograms. It's not two histograms vs. one scatter plot, it's replacing the current single histogram system with a single scatter plot.

Also, side-by-side histograms aren't the only way to display two parameters in a single histogram. What about stacked histograms? They scale up to an arbitrary number of parameters and everybody who can read a histogram can read a stacked histogram. Scatter plots seem like thermonuclear overkill for a problem which most movie sites seem to consider to be "solved".

Re: Why Ratings Systems Don't Work

#28
I'll say histograms are useful for certain things.

If you are looking at electronic devices or camera lenses there's the issue that a certain fraction of people get lemons. Some bad reviews are because of that.

Other people have unrealistic expectations of the product and give a bad review.

A histogram gives some immediate insight into this problem, and then looking at stratified samples of the reviews helps there on out.

Now, I will say the star ratings on Ebay are weak because of the fact that a less-than-perfect ranking gets people in trouble. Although "acceptable" performance on Ebay goes a considerable range (It's certainly a worse experience to have a long confused exchange with somebody with poor english -- this person shouldn't be punished, but they shouldn't be rewarded either.)

Re: Why Ratings Systems Don't Work

#29
post #19

When I'm looking for people's opinion I want to know at least a little about the people and have more things in common with them, so that we have similar tastes. IMDB is almost useless for me, people like complete crap imo (is there any movie without at least one 10 star rating? That should be the single best movie ever). If you wouldn't fit in the community, the community's opinion on things is largely irrelevant an…

You've absolutely hit the nail on the head. Knowing the rating of a friend is worth so much more than a bunch of strangers. That's what Goodfilms is all about - it puts your friends' opinions ahead.

Re: Why Ratings Systems Don't Work

#30
post #14

Earlier quoted context omitted.

It's different from having 2 histograms because it lets you distinguish people who thought it was a good movie but not re-watchable, from people who thought it was a good movie and fairly re-watchable, from people who thought it was an OK movie and not re-watchable. With a bunch of histograms you can't correlate the variables.

Why would I care about rewatchability? The torrents are full with thousands of lives on video I'll never watch, what's the point of rewatching anything but the dearest, youth-defining films?

The comment I was replying to said This would be no different than having two histograms side by side. And I pointed out that it is not.

Oh well I'll answer your question anyway. Because 90% of everything is crap (probably more than that on P2P sites). Re-watching a movie can be like walking in the same park more than once, looking at pleasant things with a sense of recognition. You could put a movie in to suit your mood. Sometimes you'd rather watch a good "original" movie again than watch yet another crappy remake or rip-off.

Post reply on HN