He shows the flaws of both examples but only gives a solution to the first one. How does one nicely sort a rating scale like Amazon's that does have more than "yes" and "no"?
How Not To Sort By Average Rating
31–34 of 34 posts
Re: How Not To Sort By Average Rating
#32A more thorough post on the same subject has been posted on HN before: How to build a popularity algorithm you can be proud of http://news.ycombinator.com/item?id=797740.
It's been a great resource for me.
EDIT: Fixed the link.
Re: How Not To Sort By Average Rating
#33He shows the flaws of both examples but only gives a solution to the first one. How does one nicely sort a rating scale like Amazon's that does have more than "yes" and "no"?
I don't know offhand how the Wilson thing generalizes. The simple Bayesian posterior mean one I mentioned yesterday is pretty simple: suppose you have a1,a2,...,ak ratings with 1,2,...,k stars; then the posterior mean number of stars is [(1+a1).1 + (1+a2).2 + ... + (1+ak).k] / [(1+a1) + (1+a2) + ... + (1+ak)]. (Of course the denominator can be written more briefly, but I think it's clearer that way.) Equivalently, you just calculate the average star count, but you add in a bunch of fictitious reviews, one with each possible number of stars. Once again, by adjusting the number of each kind of fictitious review you can (e.g.) make less-reviewed items do better or worse relative to more-reviewed ones; by changing the coefficients that are 1,2,...,k above you can (e.g.) make a 5-star review more than 5/4 as good as a 4-star one; etc.
Re: How Not To Sort By Average Rating
#34It's very nice to see a mathematically motivated piece on how to sort based on users ratings. But there are two problems with the solution presented, the first minor, and the second more serious. Minor: Distributions of ratings are not usually normal about the average possible rating -- they are usually skewed to the positive end, because people are pretty good at selecting what they read or view. The solution presen…
The article does actually cover your serious point.
> Indeed, it may be more useful in a "top rated" list to display those items with the highest number of positive ratings per page view, download, or purchase, rather than positive ratings per rating. Many people who find something mediocre will not bother to rate it at all; the act of viewing or purchasing something and declining to rate it contains useful information about that item's quality.