Why Ratings Systems Don't Work • Hacker News Board

Re: Why Ratings Systems Don't Work

#161

Post by droithomme » Thu, Aug 23, 2012, 1:43 AM UTC

vhf wrote: post #8

s_henry_paulson wrote: post #2

Terrible article. Calling histograms awful, based on nothing more than an opinion. Then trying to conclude that some convoluted scatter plot system makes more sense is laughable. Not to mention, this system is still just a star rating system. This would be no different than having two histograms side by side.. assuming, of course, that you'd even want to rate different aspects of the same thing. I can't even imagine…

This would be no different than having two histograms side by side.. Yes it would, and the article shows why and how. Scatter plots are easy to read (for comp./math. educated people). Two histograms side by side are easy to read (and find correlations) for nobody.

It's true it's harder to find correlations, but this particular correlation is not likely to be meaningful.

As far as reading the distribution, two separate histograms are definitely more readable and understandable in order to understand the distribution. Adding the complexity of a scatter plot because it also shows a correlation that people are not actually interested in makes things less understandable.

Re: Why Ratings Systems Don't Work

#162

Post by droithomme » Thu, Aug 23, 2012, 1:48 AM UTC

abrookewood wrote: post #16

vhf wrote: post #8

Earlier quoted context omitted.

This would be no different than having two histograms side by side.. Yes it would, and the article shows why and how. Scatter plots are easy to read (for comp./math. educated people). Two histograms side by side are easy to read (and find correlations) for nobody.

I'm with Henry ... the scatterplot is fundamentally no different to two histograms side-by-side (though it is marginally easier to read).

> the scatterplot is fundamentally no different

Actually it is fundamentally different as the histograms show aggregate data and scatterplots show individual data points.

Re: Why Ratings Systems Don't Work

#163

Post by droithomme » Thu, Aug 23, 2012, 1:52 AM UTC

sixbrx wrote: post #68

geelen wrote: post #3

Earlier quoted context omitted.

My point was you can't create a scatter plot from a single dimension of data, and that histograms are awful because they don't allow you to see patterns. But of course you're right, it is just my opinion.

But histograms do allow you to easily see one of the most important patterns to me, at least for technical books, which is that the book is excellent but too "hard" for some who aren't willing to put in the effort. Those books tend to have a lot of 5 star and 1 star ratings, with few in between. Check out the SICP reviews on Amazon for example. When augmented with a set of comments, it's usually easy to predict which…

I find histograms very useful on amazon. Particularly the bimodal ones, which typically indicate controversial material.

Re: Why Ratings Systems Don't Work

#164

Post by droithomme » Thu, Aug 23, 2012, 1:57 AM UTC

Stonewall9093 wrote: post #98

Interesting article but it's still arbitrary. I don't think it solves the problem still. What it boils down to is trusting the raters. This is the only way to get by spam, ignorance, and trolling. I think the solution is knowing who is rating what you're looking at. In the old sense - "Quality over quanitty." I don't need to know what the whole world gave it, just a few people who I've come to trust. We're trying to…

Yeah, it makes the bold generic claim to answer "why [all] rating systems don't work", makes the bizarre claim that histograms that show the distribution are "worse" than a scalar mean, and then presents as a solution "We've solved the generic problem by doing two things: adding a 'rewatchable' rating, and plotted it against general quality rating as a scatter plot." Rewatchable is irrelevant to things that aren't movies. The scatter plot is only relevant if the correlation between the two is important, but the given graphs don't support that the correlation has meaningful information beyond that people who like a movie are more likely to want to rewatch it, which is a correlation that is likely consistent across all movies in their dataset and conveys little information about any particular movie.

Re: Why Ratings Systems Don't Work

#165

Post by stewbrew » Thu, Aug 23, 2012, 7:40 AM UTC

Homunculiheaded wrote: post #122

I'm surprised this article, nor the discussion here doesn't address the main issue here: Just because you use numbers doesn't mean your data is quantitative. Star scores are an attempt to map a qualitative experience (enjoyment of the film) with some quantitative measure. Which is fine if you just want to get a sense of 'how much' somebody liked a something. If I say I give scotch A a 5 scotch B a 3 and scotch C a 4…

Such data is called ordinal (see https://en.wikipedia.org/wiki/Level_of_measurement#Ordinal_s...). You could use arithmetic means etc. if the distance between the categories were equal. Unfortunately, nobody defined what 1, 2, 3 ... stars mean.