Comparison is wrong between tfidf on words and CNN char. You should use char ngrams along with LR and this will beat all your classifiers with high probability. This is because your CNN char does not have enough data to draw all the useful chat ngrams. Doing it as preprocessing and passing it to LR is in practice always better on small datasets. You can go one step forward and add layers and test an MLP on your char ngrams.
How Quid uses deep learning with small data
31–40 of 49 posts
Re: How Quid uses deep learning with small data
#32There's something strange about the ROC curve here. It seems that the feature engineered and logistic regression methods can pick out some examples very easily (20% true positive rate at a very low false positive rate) but the CNN seems to not be able to make many predictions at a low false positive rate. It then catches up later. It's almost like it can't pick out the easy examples, but does just as good a job on the harder ones.
Re: How Quid uses deep learning with small data
#33> A downfall of CNNs for text is that unlike for images, the input sequences are varying sizes (i.e., varying size sentences), which means most text inputs must be “padded” with some number of 0’s, so that all inputs are the same size.
Actually Kim's model you're using doesn't require padding because it uses k-Max over time pooling.
Also kuddos for NOT updating your word embeddings during training! A lot of people are doing it, but IMHO it's a mistake most of the time.
Re: How Quid uses deep learning with small data
#34The link to LIME looks a bit out of place - LIME is an algorithm of explaining classifier decisions which is most useful for cases when you can't inspect weights and map them back to features. For TF*IDF + Logistic Regression there is no need to use LIME, one can just use weights and feature names directly. LIME is more helpful for all other models (there is a lot of caveats though), not for the basic tfidf + linear classifier model.
Re: How Quid uses deep learning with small data
#35This post is a joke. Seriously, it amazes me that the entire industry seems fixated on a handful of techniques, just like they were with random forests 10 years ago, just like they were on SVMs ten years before that, just like they were base neural networks before that. There's a simpler way, nature almost requires it.
> There's a simpler way, nature almost requires it
This is a normative statement, do you have empirical evidence?
Re: How Quid uses deep learning with small data
#36The baseline I'd like to see this compared to is the not-very-deep-learning "bag of tricks" that's conveniently implemented in fastText [1]. [1] https://github.com/facebookresearch/fastText
why not vowpal wabbit? fast text is practically a ad-hoc version of mathematically proven vowpal wabbit?
Re: How Quid uses deep learning with small data
#37Is all machine learning called "deep learning" now? Where is the line between "normal" machine learning algorithms and deep learning?
Re: How Quid uses deep learning with small data
#38Earlier quoted context omitted.
I actually had this issue recently when trying to get training data for a project of mine as well [0], so I built an app [1] as a way to more easily classify documents. Basically I have simpler interfaces and the ability for multiple people to quickly answer questions like this on a set of data. Easily exportable in the end as well. If you're interested in using that to get some more data on sentences, let me know. I…
This is really great idea. Actually if there is something you can share along these lines, that would be amazing. I know Crowd Flower has a great "internal only" tool, which is kind of similar to what you are designing, but you have to pay for it. Actually I think there is a huge need for a generic tool along the lines of what you have started to build.
Haven't heard of CrowdFlower but yeah this is along those lines. Pretty similar. But I could definitely make something quick to fit this specifically. I've been looking for other uses along with what I'm doing and this fits exactly. Shoot me an email at the address listed on my profile and I can get going.
Re: How Quid uses deep learning with small data
#39Comparison is wrong between tfidf on words and CNN char. You should use char ngrams along with LR and this will beat all your classifiers with high probability. This is because your CNN char does not have enough data to draw all the useful chat ngrams. Doing it as preprocessing and passing it to LR is in practice always better on small datasets. You can go one step forward and add layers and test an MLP on your char…
Could you elaborate. Is LR, linear regression?
Re: How Quid uses deep learning with small data
#40Comparison is wrong between tfidf on words and CNN char. You should use char ngrams along with LR and this will beat all your classifiers with high probability. This is because your CNN char does not have enough data to draw all the useful chat ngrams. Doing it as preprocessing and passing it to LR is in practice always better on small datasets. You can go one step forward and add layers and test an MLP on your char…
Thanks, would you mind expanding? I also played around with some char CNNs. They had similar performance.