I wish people would use a 5 or 7 point scale: http://blog.jgc.org/2007/12/seven-point-scale.html
Why Ratings Systems Don't Work
11–20 of 165 posts
Re: Why Ratings Systems Don't Work
#12Histograms are useful int the case of items with a strong love/hate split.
The canonical example is the SICP ratings on Amazon: 3.5 average; 177 ratings, 96 five stars, 53 one stars.
http://www.amazon.com/Structure-Interpretation-Computer-Prog...
Re: Why Ratings Systems Don't Work
#13From a statistician's standpoint, ratings systems suck. But, from a consumer standpoint, they are super easy to understand. A scatter plot system makes sense to me, but I would never put it in front of a user. In my opinion, current ratings systems are 80% UX and 20% data. For example, Newegg uses a pretty intuitive system of allowing you to sort a product page by Best Reviews and Most Reviews. In my opinion, this al…
From a consumer standpoint, a single five star review pushing a product to the top of the list is not easy to understand, it's a pain.
This a big reason we went with scatter plots - when there's only a handful of points you don't get misled.
Re: Why Ratings Systems Don't Work
#14Terrible article. Calling histograms awful, based on nothing more than an opinion. Then trying to conclude that some convoluted scatter plot system makes more sense is laughable. Not to mention, this system is still just a star rating system. This would be no different than having two histograms side by side.. assuming, of course, that you'd even want to rate different aspects of the same thing. I can't even imagine…
It's different from having 2 histograms because it lets you distinguish people who thought it was a good movie but not re-watchable, from people who thought it was a good movie and fairly re-watchable, from people who thought it was an OK movie and not re-watchable. With a bunch of histograms you can't correlate the variables.
Re: Why Ratings Systems Don't Work
#15If I'm deciding whether to watch a film for the first time, I really don't understand how whether or not other people would want to watch it a second time helps me make my decision.
"We rate movies on two criteria - ‘quality’ and ‘rewatchability’, so you can admit to your guilty pleasures and properly capture the feeling you get when a film leaves you exhausted."
You are using rewatchability to infer some potentially helpful labels ("guilty fun", "exhausting but worthy"). But there's no guarantee that those inferences are safe/generalisable across viewers/films/genres, or that people who see a rewatchable axis will know to interpret it like you do...
Re: Why Ratings Systems Don't Work
#16Terrible article. Calling histograms awful, based on nothing more than an opinion. Then trying to conclude that some convoluted scatter plot system makes more sense is laughable. Not to mention, this system is still just a star rating system. This would be no different than having two histograms side by side.. assuming, of course, that you'd even want to rate different aspects of the same thing. I can't even imagine…
This would be no different than having two histograms side by side.. Yes it would, and the article shows why and how. Scatter plots are easy to read (for comp./math. educated people). Two histograms side by side are easy to read (and find correlations) for nobody.
I'm with Henry ... the scatterplot is fundamentally no different to two histograms side-by-side (though it is marginally easier to read).
Re: Why Ratings Systems Don't Work
#17The article mentions an important point about 5 star systems - people tend to only use 5 stars or 1 star. This is sort of shown with the histograms.
A 2 axis system seems like a good idea. But I'd like to see it with 3 options per access - [UP] [INDIFFERENT] [DOWN].
I'm also interested to know how the system will cope with "controversial" films ( life of brian, for example) where some people are going to downvote whether they've seen it or not. And they'll campaign and ask all their friends to downvote too.
Re: Why Ratings Systems Don't Work
#18Terrible article. Calling histograms awful, based on nothing more than an opinion. Then trying to conclude that some convoluted scatter plot system makes more sense is laughable. Not to mention, this system is still just a star rating system. This would be no different than having two histograms side by side.. assuming, of course, that you'd even want to rate different aspects of the same thing. I can't even imagine…
It's different from having 2 histograms because it lets you distinguish people who thought it was a good movie but not re-watchable, from people who thought it was a good movie and fairly re-watchable, from people who thought it was an OK movie and not re-watchable. With a bunch of histograms you can't correlate the variables.
Why would I care about rewatchability? The torrents are full with thousands of lives on video I'll never watch, what's the point of rewatching anything but the dearest, youth-defining films?
Re: Why Ratings Systems Don't Work
#19When I'm looking for people's opinion I want to know at least a little about the people and have more things in common with them, so that we have similar tastes. IMDB is almost useless for me, people like complete crap imo (is there any movie without at least one 10 star rating? That should be the single best movie ever). If you wouldn't fit in the community, the community's opinion on things is largely irrelevant and whether you look at the opinion through a histogram or a scatter plot is irrelevant.
If you actually have friends, why don't you ask them for recommendation in person. If your friend is really into arty movies and recommends you an arty movie as being very arty (and well done) you can consider it. Collapsing it into a single number doesn't make sense
EDIT: that's not to say that the scatter plot isn't an interesting idea, it's just not going to help much because people's background is important for rating
Re: Why Ratings Systems Don't Work
#20"People like you rated it..." is much better for users than two axes.