Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask/Poll News.YC: What is a good open source Bayes classifier?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 41 posts

Re: Ask/Poll News.YC: What is a good open source Bayes classifier?

#14

Post by fiaz »

jgrahamc wrote: post #2

What sort of classification are you trying to do? Text, I assume. If it's text and you need something open source, or just a pointer to how to write one yourself, you could read the article I wrote for Dr Dobbs on this subject: http://www.ddj.com/development-tools/184406064 There are quite a lot of toolkits out there that do Bayesian things (take a look at libbow or Weka).

I'm looking for something that is open source that I can start using right away with minimal configuration... I'm not smart enough to create my own, but clever enough to take somebody else's work and cater it to what I'm trying to do.

It's only for text processing.

Re: Ask/Poll News.YC: What is a good open source Bayes classifier?

#18

Post by dreish »

jgrahamc wrote: post #6
dreish wrote: post #5

Whichever one you use, you'll probably need to edit the source to change the probability formula from (n_hits / n_total) to ((n_hits + 1) / (n_total + 2)). That's the correct formula based on an even distribution of probabilities (which is close enough to the actual distribution in most situations for this to be a huge improvement). I can never find a reference for this when I search for one, but you can verify it ex…

Sounds to me that you are describing Laplace Smoothing in a Naive Bayes classifier. This is a pretty standard technique for avoiding the problem that you are seeing where the probability comes out as 100%/0% because of a lack of information in the model.

Thanks, yes, that's what it's called.

All due credit to Laplace for the technique, but the word "smoothing" is making me wince, because it makes it sound as though this is some artificial approximation. For the assumption of an even distribution of probabilities, n+1 / m+2 really _is_ the exact probability of the event repeating. Like I said, you can confirm this experimentally with a quick program.

Post reply on HN