101–110 of 159 posts
Original author here. For the academically inclined, there is a critique of this approach in this paper: http://www.dcs.bbk.ac.uk/~dell/publications/dellzhang_ictir2... Of course, I think the authors miss the point of the algorithm, since I basically wanted a system that is one-sided (i.e. false negatives are OK but false positives are bad). Also, if you deal with more than two outcomes you might be interested in mul…
Earlier quoted context omitted.
I would absolutely love to learn more. I've been trying to solve some novel NLP and machine learning problems lately but my lack of statistical knowledge is becoming apparent the further along I get. Do you have any recommendations for a good introductory treatment of Bayesian statistics?
By far my favorite book on the subject is conveniently available for free on the internet! "Information Theory, Inference and Learning Algorithms" by David MacKay is wonderfully written, well paced and comprehensive. If you like the book, you should purchase a copy, David is a great guy. http://www.inference.phy.cam.ac.uk/mackay/itila/book.html Videolectures has some very good videos as well. Zoubin Gharamani has a p…
His book starts from first principles -- simple ideas about probabilities -- and it builds a foundation for understanding Bayesian methods. And he explains, basically, why the p-values and t-test stuff is a bunch of crap, which was an immense relief.
Earlier quoted context omitted.
>What human can understand it? Lets start with Wilson's midpoint, since that's just high school math. def mid(upvotes:Int, downvotes:Int) = { val total = upvotes+downvotes+0.0 val up = upvotes/total val half = 0.5 val a = total/(4+total) val b = 4/(4+total) a * up + b * half } So there are two weights a and b. Using these weights, the midpoint is a weighted average of half and the proportion of upvotes. It should be…
I think the original post still stands - try explaining the above to an irate customer.
While I agree with the spirit of the article, this is one of those cases where a Bayesian treatment is conceptually much clearer. Assume that ratings are being generated by a stable stochastic process where the underlying distribution is multinomial (ignoring the ordinal character of ratings, for the time being) and use a dirichlet conjugate prior. This gives you a posterior distribution over new ratings for an item.…
You could also go one step further on the Bayesian path and infer even alpha from the data on your site, and introduce a loss function on your ordering. Or you could do a semi-frequentist thing and simplify your math by using MAP estimates to rank. Basically instead of score = #pos/(#pos + #neg), it becomes score = (#pos+x)/(#pos+x + #neg+y), where you choose x and y to suit your needs. You could choose x/y in propor…
I love it and I hate it. Why I love it: It's precise. It's elegant. It's rigorous. It's based upon solid, proven science & theory. It's a perfect application for a computer. And most of all, it does what's intended: it works. Why I hate it: What human can understand it? I used to implement the first manufacturing and distribution systems that used thinking like this. They figured, "We finally have the horsepower to a…
Urban Dictionary no longer sorts by positive - negative, see e.g. http://www.urbandictionary.com/define.php?term=usa . I don't know what they use now.
"USA: The only country keeping penguins from coquering [sic] the Earth"
Original author here. For the academically inclined, there is a critique of this approach in this paper: http://www.dcs.bbk.ac.uk/~dell/publications/dellzhang_ictir2... Of course, I think the authors miss the point of the algorithm, since I basically wanted a system that is one-sided (i.e. false negatives are OK but false positives are bad). Also, if you deal with more than two outcomes you might be interested in mul…
Mathematician not a statistician... Would it be reasonable for 5 stars to normalise the data? Should star ratings be on some distribution, for instance?
This is a little flakey for ordinal numbers, and the usual treatment is to use a learning algorithm to find a mapping from real numbers to ordinal values, either explicitly (if you need a "score") or implicitly. Support vector machines, radial basis functions and neural networks are typically used.
This must be the third time this has been posted.
Earlier quoted context omitted.
> It just requires better educators to explain it, with analogies and metaphor. So... anyone want to take a stab at explaining that equation to those of us who don't really get it?
If a comment has one upvote and zero downvotes, it has a 100% upvote rate, but since there's not very much data, the system will keep it near the bottom. But if it has 10 upvotes and only 1 downvote, the system might have enough confidence to place it above something with 40 upvotes and 20 downvotes -- figuring that by the time it's also gotten 40 upvotes, it's almost certain it will have fewer than 20 downvotes. And…