What Kagglers Are Using for Text Classification
1–10 of 72 posts
Re: What Kagglers Are Using for Text Classification
#2This was really interesting and insightful thanks!
Re: What Kagglers Are Using for Text Classification
#3It would be nice to see how these methods compare to the classical methods based on word occurrences.
Re: What Kagglers Are Using for Text Classification
#4Three methods and no idea how I would choose between the three of them aside randomly trying each one and measuring performance.
(Not for winning kaggle but for an actual problem)
Re: What Kagglers Are Using for Text Classification
#5It would be nice to see how these methods compare to the classical methods based on word occurrences.
Kaggle is a pretty serious natural-selection environment for machine learning algorithms. Basically, if bag-of-words worked better, the contest winners would still use it.
Re: What Kagglers Are Using for Text Classification
#6Three methods and no idea how I would choose between the three of them aside randomly trying each one and measuring performance. (Not for winning kaggle but for an actual problem)
Sometimes you just need to rent 3 separate EC2 instances and try precisely that.
Re: What Kagglers Are Using for Text Classification
#7I wonder how FastText (essentially word2vec + word & char n-grams + other stuff) stacks up against these algorithms.
In my own tests on my own corpuses, CPU-based FastText is faster to train and produces significantly better results (precision/recall) than the GPU-bound CNN algorithms that I've tried, but have not compared it against RNN techniques.
Re: What Kagglers Are Using for Text Classification
#8I wonder how FastText (essentially word2vec + word & char n-grams + other stuff) stacks up against these algorithms. In my own tests on my own corpuses, CPU-based FastText is faster to train and produces significantly better results (precision/recall) than the GPU-bound CNN algorithms that I've tried, but have not compared it against RNN techniques.
[deleted]
Re: What Kagglers Are Using for Text Classification
#9It would be nice to see how these methods compare to the classical methods based on word occurrences.
Kaggle is a pretty serious natural-selection environment for machine learning algorithms. Basically, if bag-of-words worked better, the contest winners would still use it.
One issue is the kind of problem.
I remember getting 95%-ish accuracy with BoW and the SVM circa 2004 when it came to questions like "is this paper about astrophysics or organic chemistry?"
In that case you have a distinct vocabulary for different topics and it is hard to beat BoW.
Sentiment analysis, on the other hand, is where BoW goes to die since now "not good" means something very different than "good", and even simple heuristics like treating "not X" as a term that is different from "X" give limited gain because negation is expressed with constructions like "i don't believe that is good" and there is no k-word window that you reliably catch negation in since there isn't a limit on how complex sentences are.
There is also the question of "is the improvement between method A and method B worth it?" For instance the Netflix prize was much celebrated because some brilliant people busted their ass to go from 92% to 95% accuracy on movie recommendations. In the end the algorithm proved to be too complex for the value it created. (eg. Who would notice that they got 8 bad recommendations instead of 5 out of a hundred? An additional half a bad recommendation out of 10?)
The real "Netflix optimization problem" is how to spend as little on acquiring content as possible while motivating people to keep their subscriptions and that is something Netflix will keep closer to their chest and not promote a public competition on. (eg. if it were valuable why would they let competitors know about it?)
Re: What Kagglers Are Using for Text Classification
#10It would be nice to see how these methods compare to the classical methods based on word occurrences.
Kaggle is a pretty serious natural-selection environment for machine learning algorithms. Basically, if bag-of-words worked better, the contest winners would still use it.
It is important to understand "worked better". In practice, I would trade off using simpler methods for few points increase in precision/recall/your choice of metric.