11–20 of 62 posts
Previously: https://news.ycombinator.com/item?id=478632
Some of the comments from that posting give concrete examples where the formula fails. Such as: an item with 1000 upvotes and 2000 downvotes will get ranked above one with 1 upvote and 2 downvotes. This is because the formula uses the lower bound of the Wilson interval.
There is a much simpler and elegant method. Just rank posts by their probability of getting an upvote. This is just (upvotes+1)/(upvotes+downvotess+2).
Earlier quoted context omitted.
I agree. It's a good solution for all cases where the intent is to have a negative vote exactly cancel out a positive. The method in the article combines a quality rating with a quantity rating, but it's a bit unwieldy and difficult to tune intuitively. It seems to me for a lot of purposes you might get a sufficiently similar effect by using method #1, and then multiplying the result with the sigmoid function applied…
YMMV, but I'd rather use an elementary and mathematically sound statistical technique than macguyvering something myself. (Though I do understand that, as some people describe in this thread, there can be different purposes to ratings and hence a need for different sorting mechanisms.) You're right that confidence intervals depend on both quality and quantity, but the reason for this is to account for uncertainty. As…
You're using a highly biased sample, to begin with. The mathematics here start with the assumption that you have a random sample. You don't. The assumption is invalid; and there's no reason to believe this calculation is a good one compared with any other.
A particular type of person votes on particular things: it's commonly observed that new movies rate higher than they should on IMDB, because super-fans are the first to vote. (This is convenient for your example)
I don't think the HN algorithm is public, but wouldn't be surprised if it does the same.
Perhaps the generally much smaller number of 'votes' on a HN/reddit post makes it less significant.
Interestingly, I _think_ the Reddit algorithm basically makes this mistake too -- although embedded in a more complicated algorithm that combines with 'newest first' altered by positives minus negatives. I don't think the HN algorithm is public, but wouldn't be surprised if it does the same. Perhaps the generally much smaller number of 'votes' on a HN/reddit post makes it less significant.
You only get this ranking method if you sort the comments by 'best' though
Also discussed in Cameron Davidson-Pilon's Bayesian methods for Hackers in the context of Reddit ups/downs: http://nbviewer.ipython.org/github/CamDavidsonPilon/Probabil... For Amazon, though, which is the example in Evan Miller's post, I don't really get why you'd first dichotomize the five-star rating into positive vs. negative and then use Wilson intervals. Just construct a run-of-the-mill 95% confidence interval f…
These companies really should be hiring statistics consultants instead of relying on the intuitions of their programmers.
Earlier quoted context omitted.
YMMV, but I'd rather use an elementary and mathematically sound statistical technique than macguyvering something myself. (Though I do understand that, as some people describe in this thread, there can be different purposes to ratings and hence a need for different sorting mechanisms.) You're right that confidence intervals depend on both quality and quantity, but the reason for this is to account for uncertainty. As…
Using internet points this way is not mathematically sound. You're using a highly biased sample, to begin with. The mathematics here start with the assumption that you have a random sample. You don't. The assumption is invalid; and there's no reason to believe this calculation is a good one compared with any other. A particular type of person votes on particular things: it's commonly observed that new movies rate hig…
You make a good point that there might be differential bias depending on when the movie came out, but I don't think the solution is then to say "well, now all bets are off, might as well concoct our own techniques and assume they're just as good or better." Statistical techniques are not either fully valid or fully invalid. Simulate the bias and look at exactly how it influences the results of a particular technique, and then perhaps use that to make an adjustment based on data rather than intuition.