Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Not To Sort By Average Rating

evanmiller.org

Post reply on HN
Print view View original

11–20 of 159 posts

Re: How Not To Sort By Average Rating

#11

Post by tel »

edw519 wrote: post #10

I love it and I hate it. Why I love it: It's precise. It's elegant. It's rigorous. It's based upon solid, proven science & theory. It's a perfect application for a computer. And most of all, it does what's intended: it works. Why I hate it: What human can understand it? I used to implement the first manufacturing and distribution systems that used thinking like this. They figured, "We finally have the horsepower to a…

I hate it exclusively. It's clearly better than (1) and (2), but...

I talk about this with some of my med school friends that are interested in/want to create/hate/fear automated diagnosis. At one level, having appropriate statistics to make use of a wealth of prior experience, worldwide prevalence, epidemiological data, &c is basically a requirement for the future of proper healthcare.

It's also an obvious terrible mistake to ever convince people that these statistics are anything besides a decision making tool.

I think the right usage of statistics is to enlighten and confuse simultaneously, not to "answer". They should provide analytical depth to a decision, never an escape route.

For this reason, I think proper statistical application is an intersection between mathematics, computer science, and UI design.

Re: How Not To Sort By Average Rating

#12

Post by omarqureshi »

The one example that I've found of a good site that does really good average ratings is steepster, it picks teas that you have previously rated and indicates the rating you gave to them. This way the users rating is much better and will give you a much more meaningful mean.

Re: How Not To Sort By Average Rating

#13

Post by raldi »

yariang wrote: post #5

While it is good to look at these sorts of mathematically rigorous algorithms, I think I would be frustrated if it was used everywhere. Or, well, maybe not me perhaps, but a non technical user. The beauty of the second algorithm for rating products is that it is straightforward. Having never seen it before I can deduce that 5 stars come before 4 stars and more reviews come before fewer. If I want to skip ahead to the…

Instead of displaying stars, Amazon could display a percentage, which under the hood represents the Wilson confidence number. It would be totally intuitive to browse: first come all the 100% items, then the 99's, and so on.

Re: How Not To Sort By Average Rating

#14

Post by jwr »

I implemented this in a rating system once. Got multiple bug reports, people complained that the system calculates averages wrong, because there are two ratings and the average is obviously not the number they are seeing.

Re: How Not To Sort By Average Rating

#15

Post by edash »

jwr wrote: post #14

I implemented this in a rating system once. Got multiple bug reports, people complained that the system calculates averages wrong, because there are two ratings and the average is obviously not the number they are seeing.

Why show the second figure to the user? They don't need to see the calculation you made to determine the sort.

Just sort the objects in the order determined by this formula and only show the ratings given by users in the interface.

Re: How Not To Sort By Average Rating

#17

Post by m_for_monkey »

edw519 wrote: post #10

I love it and I hate it. Why I love it: It's precise. It's elegant. It's rigorous. It's based upon solid, proven science & theory. It's a perfect application for a computer. And most of all, it does what's intended: it works. Why I hate it: What human can understand it? I used to implement the first manufacturing and distribution systems that used thinking like this. They figured, "We finally have the horsepower to a…

You have overlooked a critical point: no one has to mess with this (otherwise quite simple) formula anymore, except the programmer, and even he already got a working piece of Ruby code.

Re: How Not To Sort By Average Rating

#18

Post by imajes »

edw519 wrote: post #10

I love it and I hate it. Why I love it: It's precise. It's elegant. It's rigorous. It's based upon solid, proven science & theory. It's a perfect application for a computer. And most of all, it does what's intended: it works. Why I hate it: What human can understand it? I used to implement the first manufacturing and distribution systems that used thinking like this. They figured, "We finally have the horsepower to a…

what?

There's nothing wrong with the math. It just requires better educators to explain it, with analogies and metaphor. Why would you compromise the data/outcome in a hope to simplify the problem?

Further: wouldn't you look to hire (and train) the best people who understand the domain they are in, thereby being able to judge whether the outcome of an equation is valid or not?

Hint: insurance/loan underwriters regularly end up in situations where the human component of a transaction may look different than the data, and react accordingly...

Re: How Not To Sort By Average Rating

#19

Post by imajes »

raldi wrote: post #13
yariang wrote: post #5

While it is good to look at these sorts of mathematically rigorous algorithms, I think I would be frustrated if it was used everywhere. Or, well, maybe not me perhaps, but a non technical user. The beauty of the second algorithm for rating products is that it is straightforward. Having never seen it before I can deduce that 5 stars come before 4 stars and more reviews come before fewer. If I want to skip ahead to the…

Instead of displaying stars, Amazon could display a percentage, which under the hood represents the Wilson confidence number. It would be totally intuitive to browse: first come all the 100% items, then the 99's, and so on.

er... and the problem with bucket categorizing?

80-100% = * * * * *

60%-79% = * * * *

etc..

Re: How Not To Sort By Average Rating

#20

Post by aw3c2 »

gipsyking wrote: post #7
aw3c2 wrote: post #2

Every time I see that page, I see the equation, I read statistical terms and I get overwhelmed. I use PHP so I have no pnormaldist. Would love to use it for some random page I run.

"if you don't have a statistics package handy or if performance is an issue you can always hard-code a value here for z. (Use 1.96 for a confidence level of 0.95.)"

Ouch, I managed not to see that at all. Thanks!
Post reply on HN