really nice article -- easy to follow, sensible steps, clean code. I haven't done too much text ML and this was a nice piece to follow - thanks!
How Quid uses deep learning with small data
21–30 of 49 posts
Re: How Quid uses deep learning with small data
#22This post is a joke. Seriously, it amazes me that the entire industry seems fixated on a handful of techniques, just like they were with random forests 10 years ago, just like they were on SVMs ten years before that, just like they were base neural networks before that. There's a simpler way, nature almost requires it.
Re: How Quid uses deep learning with small data
#23If you're interested in Sequence to Sequence tasks (e.g. neural machine translation or abstractive summarization) with small data, check out our recent paper from Google Brain tackling this problem (disclaimer, I'm the first author): https://arxiv.org/abs/1611.02683
Thanks! I'll check it out. I have also been reading about abstractive summarization - hard problem it seems!
Re: How Quid uses deep learning with small data
#24Earlier quoted context omitted.
Thanks for the comment! There were a few hundred sentences of each, collected internally from from a wide number of descriptions. Yes, I'd definitely agree- more data is what we need here for further model improvements.
I actually had this issue recently when trying to get training data for a project of mine as well [0], so I built an app [1] as a way to more easily classify documents. Basically I have simpler interfaces and the ability for multiple people to quickly answer questions like this on a set of data. Easily exportable in the end as well. If you're interested in using that to get some more data on sentences, let me know. I…
This is really great idea. Actually if there is something you can share along these lines, that would be amazing. I know Crowd Flower has a great "internal only" tool, which is kind of similar to what you are designing, but you have to pay for it. Actually I think there is a huge need for a generic tool along the lines of what you have started to build.
Re: How Quid uses deep learning with small data
#25Wait, isn't "deep learning with small data" just machine learning, after all the buzzwords cancel themselves out? I thought the whole point of "deep learning" is its approach to using data.
The point of deep learning is using a deep graph, like a neural network with a lot of layers, not the amount of data. However, picking millions of parameters with small amounts of data is unlikely to work well.
I would agree. Its kind of amazing that it indeed it does seem to work better than simpler models.
Re: How Quid uses deep learning with small data
#26I can't seem to find where the sample size is mentioned. It mentions that Quid has 50,000 company descriptions, but is n=50,000 tiny in thr ML/DeepLearning world? I do neuroscience research and where I am coming from I have maybe n=150to200 per class. And that is not generally regarded as a tiny sample.
50K is not that tiny, but I'd say in the hundreds is pretty small. I'd say hundreds of training examples is pretty normal in academia, but typically quite small in industry, particularly for problems that are actually quite abstract, such as this (sentiment is another similar problem).
Re: How Quid uses deep learning with small data
#27Why do you not have a dev dataset? Gridsearch over your test dataset is bound to overfit
Re: How Quid uses deep learning with small data
#28about that detecting generic text that conveys little information
Can I have that for my email? (Seriously) And as browser plugin? Oh and on telephone, TV, radio and in real-life would be also nice.
It's probably also a nice predictor of startup success, developer quality and sales guy effectiveness.
I just wonder if I would ever read or hear a Politician again.
Very inspiring...
Re: How Quid uses deep learning with small data
#29Since word embeddings were the starting point, I'm wondering what would the impact be if they'd stretched the vector sequences to the same length using linear or whatever interpolation as opposed to zero padding the sentences.
Re: How Quid uses deep learning with small data
#30This is really great idea. Please check more about deep learning http://buff.ly/2f9UXZn