151–159 of 159 posts
Earlier quoted context omitted.
I really don't think that solves the problem; you look at product X, it has 0 reviews, you give it a 5 star, now it shows 1 rating and 3.7 stars?
So you let people rate it with 5 stars, but then show it as 3.7 magic points. Magic points can be your proprietary ranking system. You don't need to let them rate with stars and also show it as stars.
While I agree with the spirit of the article, this is one of those cases where a Bayesian treatment is conceptually much clearer. Assume that ratings are being generated by a stable stochastic process where the underlying distribution is multinomial (ignoring the ordinal character of ratings, for the time being) and use a dirichlet conjugate prior. This gives you a posterior distribution over new ratings for an item.…
- Considering the standard deviation of ratings. On a 5 point scale, an item that rates 3 because ratings are split between 1 and 5 votes, differs from one that gets mostly 3 votes. The latter is a middlin' fit for anyone, the former has an enthusiastic but niche audience. If you're looking at sales, the former can be a valuable product if properly marketed.
- An item that gathers few votes regardless of favorability ratings can exhibit multiple problems. One is that it isn't well marketed / publicised, or known. Another (particularly on content sites) is that there's very likely a sampling bias (mutual admiration society / negging attack / vote stuffing). I've tended to favor systems which take into account the total volume of voting, generally on a ln(n) basis, though not out of any particular statistical rigor. As an implementation, you'd start with a 5 point Likert score, then multiply by, say, ln(n+1) (avoiding a zero multiplier on a single vote).
- The pattern of ratings over time and space (IP or geographical) may reveal both opportunities for marketing and/or issues with your ratings system. Since any effective quality proxy will be abused, you've got to be sensitive to the latter.
The Wilson score is an improvement over multiple other methods. It still does assume a relatively unbiased estimator and rating behavior. My feeling and experience is that excess reliance on any one metric is likely to cause problems -- reality is multidimensional, metrics for assessing reality should be as well.
There's also the question of whether or not you want to make specific recommendations for an individual, or general recommendations for a population. In the former case, correlating other rankings or behavior may give a better fit (and the Wilson score may still be useful).
Though for a suitably specific goal (marketing, suitability, revenue potential) a single encompassing metric may work.
It's well explained, informally. The giant equation sitting there without clearly defined parameters is mostly just showing off though. The final "QED" solution that you put at the end of a paper is not the proper form to introduce a concept. But... so what? Amazon and Urban Dictionary are hardly failing in the market due to their "incorrect" score sorting. The whole problem is a heuristic, it's not amenable to rigor…
It's been fine in those contexts, but imagine if HN ranked +1/-0 above +100/-1
This must be the third time this has been posted.
The last time was about a month ago IIRC. Can anyone explain why this keeps happening (and why people keep giving karma to those who just repost month old HN links)?
I love it and I hate it. Why I love it: It's precise. It's elegant. It's rigorous. It's based upon solid, proven science & theory. It's a perfect application for a computer. And most of all, it does what's intended: it works. Why I hate it: What human can understand it? I used to implement the first manufacturing and distribution systems that used thinking like this. They figured, "We finally have the horsepower to a…
This may or may not be a problem. Remember, human intuition evolved to be able to give a fast answer to any question in the face of insufficient data, not an optimal answer to specific questions with sufficient data. In many domains, once you have appropriate data and the means to statistically analyze it, an algorithm using such analysis can reliably outperform human experts - and in some cases, the problem was the political one of how to get the human experts to accept this, take their hands off the wheel and stop second-guessing the algorithm.
In other words, if you are dealing with such a domain, you may be better off to count your blessings, let the computer do its job and save the humans for tasks that can't be handled by a formula.
Earlier quoted context omitted.
By far my favorite book on the subject is conveniently available for free on the internet! "Information Theory, Inference and Learning Algorithms" by David MacKay is wonderfully written, well paced and comprehensive. If you like the book, you should purchase a copy, David is a great guy. http://www.inference.phy.cam.ac.uk/mackay/itila/book.html Videolectures has some very good videos as well. Zoubin Gharamani has a p…
I want to second MacKay's book. I had a terrible statistics class in college. We spent the entire time looking up tables of p-values and t tests, without a very convincing explanation as to why. The entire topic was damaged for me from then on until I read MacKay online, and then bought the paper version. His book starts from first principles -- simple ideas about probabilities -- and it builds a foundation for under…
Thanks for the resource, this looks fantastic.
Earlier quoted context omitted.
I would absolutely love to learn more. I've been trying to solve some novel NLP and machine learning problems lately but my lack of statistical knowledge is becoming apparent the further along I get. Do you have any recommendations for a good introductory treatment of Bayesian statistics?
By far my favorite book on the subject is conveniently available for free on the internet! "Information Theory, Inference and Learning Algorithms" by David MacKay is wonderfully written, well paced and comprehensive. If you like the book, you should purchase a copy, David is a great guy. http://www.inference.phy.cam.ac.uk/mackay/itila/book.html Videolectures has some very good videos as well. Zoubin Gharamani has a p…