How Quid uses deep learning with small data
1–10 of 49 posts
Re: How Quid uses deep learning with small data
#2The baseline I'd like to see this compared to is the not-very-deep-learning "bag of tricks" that's conveniently implemented in fastText [1].
Re: How Quid uses deep learning with small data
#3What's the difference between softmax with categorical loss ([0, 1]) and sigmoid binary loss? ([0/1])
Re: How Quid uses deep learning with small data
#4What about FastText?
Re: How Quid uses deep learning with small data
#5I can't seem to find where the sample size is mentioned. It mentions that Quid has 50,000 company descriptions, but is n=50,000 tiny in thr ML/DeepLearning world?
I do neuroscience research and where I am coming from I have maybe n=150to200 per class. And that is not generally regarded as a tiny sample.
Re: How Quid uses deep learning with small data
#6I can't seem to find where the sample size is mentioned. It mentions that Quid has 50,000 company descriptions, but is n=50,000 tiny in thr ML/DeepLearning world? I do neuroscience research and where I am coming from I have maybe n=150to200 per class. And that is not generally regarded as a tiny sample.
The issue is those 50,000 descriptions aren't labeled good/bad. Someone had to pick a subset of them and label them, so my guess is they did this for maybe 100 or 200 descriptions.
Re: How Quid uses deep learning with small data
#7I can't seem to find where the sample size is mentioned. It mentions that Quid has 50,000 company descriptions, but is n=50,000 tiny in thr ML/DeepLearning world? I do neuroscience research and where I am coming from I have maybe n=150to200 per class. And that is not generally regarded as a tiny sample.
n=50000 of tabular data is a good sample size, and results will likely have a low standard error assuming no systemic bias. (Although it's not "big" data)
n=50000 of text data is different, since there will be less repetition of contextual structures and words (particularly with proper nouns). The fact that the dataset only uses "hundreds" as mentioned in the original post is interesting.
Re: How Quid uses deep learning with small data
#8Curious how you guys got training data for this. Did someone have to go through and rate whether or not a sentence was quality or not? And how many training examples did you use? You say it was "difficult to develop a large set" but I'm curious how large that set actually was.
Edit: Also, do you think more data or a "better" or "more sophisticated" model would make the results better? I would guess more data would trump better model, but not sure.
Re: How Quid uses deep learning with small data
#9Curious how you guys got training data for this. Did someone have to go through and rate whether or not a sentence was quality or not? And how many training examples did you use? You say it was "difficult to develop a large set" but I'm curious how large that set actually was. Edit: Also, do you think more data or a "better" or "more sophisticated" model would make the results better? I would guess more data would tr…
Thanks for the comment! There were a few hundred sentences of each, collected internally from from a wide number of descriptions.
Yes, I'd definitely agree- more data is what we need here for further model improvements.
Re: How Quid uses deep learning with small data
#10The baseline I'd like to see this compared to is the not-very-deep-learning "bag of tricks" that's conveniently implemented in fastText [1]. [1] https://github.com/facebookresearch/fastText
Great point! I considering using fasttext as a baseline, however in practice fasttext really didn't work well at all with the small data set, much worse than the tfidf baseline. I think Fasttext's classification approach might not work well with such a small dataset. I'm not sure but I suspect its because it tries to learn embeddings - but there just isn't anywhere near enough data for that. I'd love an outside perspective on this.