One simple thing to do would be to download CRM-114 and use it.
Ask YC: How would you do keyword classification
11–20 of 21 posts
Re: Ask YC: How would you do keyword classification
#12Check out our tool, http://tagger.flaptor.com
It's based on a Bayesian algorithm plus a bunch of other heuristics for fine-tuning. In our case, the classification algorithm is not nearly as important as how we select documents for the training set.
Also, take a look at this post that was mentioned here a few days ago:
More data usually beats better algorithms: http://anand.typepad.com/datawocky/2008/03/more-data-usual.h...
Re: Ask YC: How would you do keyword classification
#13One simple thing to do would be to download CRM-114 and use it.
See here for more on CRM-114:
Re: Ask YC: How would you do keyword classification
#14When you refer to classification, are you really talking about clustering? Is it the same thing? I've been looking at document/article clustering for NewsCred, and so far have spent some time with Carrot. Its open source and pretty good...
Re: Ask YC: How would you do keyword classification
#15When you refer to classification, are you really talking about clustering? Is it the same thing? I've been looking at document/article clustering for NewsCred, and so far have spent some time with Carrot. Its open source and pretty good...
Hi there. I've not heard of Carrot. Do you have a link for it? My Google search brought up some orange things you grow and eat :)
Re: Ask YC: How would you do keyword classification
#16Have a look at Yahoo's Content Analysis Term Extraction Web Service: http://developer.yahoo.com/search/content/V1/termExtraction....
Re: Ask YC: How would you do keyword classification
#17i think you are looking for document classification algorithms: http://en.wikipedia.org/wiki/Document_classification the current state of the art algorithms are based on support vector machines, but their learning part could be tricky to implement in a scalable fashion. if you are looking for a quick and dirty approach, TFIDF algorithm (it is a naive "naive Bayes" :) is simple and is adequate for many applications
I think every language has an open source naive Bayes/Bayes network implementation. And most of the time they are good enough.
KVM (support vector machines) so far is considered the best classification algorithm.
Re: Ask YC: How would you do keyword classification
#18i think you are looking for document classification algorithms: http://en.wikipedia.org/wiki/Document_classification the current state of the art algorithms are based on support vector machines, but their learning part could be tricky to implement in a scalable fashion. if you are looking for a quick and dirty approach, TFIDF algorithm (it is a naive "naive Bayes" :) is simple and is adequate for many applications
I think every language has an open source naive Bayes/Bayes network implementation. And most of the time they are good enough. KVM (support vector machines) so far is considered the best classification algorithm.
>KVM (support vector machines) so far is considered the best classification algorithm.
In the spirit of accuracy, SVM algorithms aren't _the best_. The best algorithms are ensemble-based, incorporating SVM and alternatives.
Re: Ask YC: How would you do keyword classification
#19Earlier quoted context omitted.
forgot to mention that you may want to look at the Orange framework which is in Python http://www.ailab.si/orange/
great! that does look interesting. I'll have a good look through it. You involved in this field yourself? If so, drop me a line as we're looking for a consultant to help us with something
This sounds interesting to me too, and in an area where I've had some experience. If nothing else perhaps we can trade notes. You'll find contact details in my profile.
Re: Ask YC: How would you do keyword classification
#20When you refer to classification, are you really talking about clustering? Is it the same thing? I've been looking at document/article clustering for NewsCred, and so far have spent some time with Carrot. Its open source and pretty good...
clustering != classification
- classification (also known as categorization) - you have a set of categories into which you expect your documents to be assigned to (the documents will be matched with the existing categories)
- clustering - you expect the algorithm to give you a set of categories (the ML algorithms will find similarities between the several documents and will group them accordingly)