Man that video at the bottom is great. I just finished up research on an image classification task in medical imaging and that tool would have helped out a lot in debugging and interpreting results - especially when you're working on image datasets of objects which you aren't very used to (like medical datasets where different tissue textures are important and only medical experts can distinguish between them).
State-of-the-art text classification with universal language models
11–20 of 54 posts
Re: State-of-the-art text classification with universal language models
#12I've yet to see data this is beating a SVM on a 4-character shingle approach (which also doesn't require tons of data to train).
Re: State-of-the-art text classification with universal language models
#13Fantastic! Thanks for sharing this! Can't wait to get inspiration from the paper to my work! ;-)
Re: State-of-the-art text classification with universal language models
#14I'm glad we're again concentrating on newer language models.
Curious how it'll perform compared to fasttext when used as encoding network in larger tasks. I can't help but notice the trend of going back to simpler models with smarter optimizations and regularization to achieve better results.
This is a frequent question of mine, which I ask to everyone using RNNs - what do you think of the idea that CNNs will be able to replace RNNs for sequence tasks [0]? CNNs are less computationally expensive too, so there's a definite benefit of switching to them if the performance is on par.
Re: State-of-the-art text classification with universal language models
#15I'm glad we're again concentrating on newer language models. Curious how it'll perform compared to fasttext when used as encoding network in larger tasks. I can't help but notice the trend of going back to simpler models with smarter optimizations and regularization to achieve better results. This is a frequent question of mine, which I ask to everyone using RNNs - what do you think of the idea that CNNs will be able…
fasttext is just an encoding of the first layer of a model (the word embeddings - or subword embeddings). Full multi-layer pre-trained models are able to do a lot more. For instance, on IMDb sentiment our method is about twice as accurate as fasttext.
As to whether CNNs can replace RNNs in general, the jury is still out. Over the last couple of years there have been some sequence tasks where CNNs are state of the art, some where RNNs are. Note that with stuff like QRNNs the assumption that CNNs are less computationally expensive is no longer necessarily true: https://github.com/salesforce/pytorch-qrnn
I'd be surprised if for tasks that require long-term state (like sentiment analysis on large docs) whether CNNs will win out in the end, since RNNs are specifically designed to be stateful - especially with the addition of an attention layer.
Re: State-of-the-art text classification with universal language models
#16Re: State-of-the-art text classification with universal language models
#17Jeremy here (co-author of this paper). Let me know if you have any questions!
Is this the same classification project that was presented at this past Bsides NoVA?
Re: State-of-the-art text classification with universal language models
#18I'm glad we're again concentrating on newer language models. Curious how it'll perform compared to fasttext when used as encoding network in larger tasks. I can't help but notice the trend of going back to simpler models with smarter optimizations and regularization to achieve better results. This is a frequent question of mine, which I ask to everyone using RNNs - what do you think of the idea that CNNs will be able…
fasttext is just an encoding of the first layer of a model (the word embeddings - or subword embeddings). Full multi-layer pre-trained models are able to do a lot more. For instance, on IMDb sentiment our method is about twice as accurate as fasttext. As to whether CNNs can replace RNNs in general, the jury is still out. Over the last couple of years there have been some sequence tasks where CNNs are state of the art…
Just remembered I already asked you about the CNNs, my bad!
Oh wow, didn't realize that these were multi-layer pre-trained models.
Also, started going through the QRNNs, they mention they've updated the AWD-LSTM Language model to use QRNNs, which is what your paper uses!
Re: State-of-the-art text classification with universal language models
#19Re: State-of-the-art text classification with universal language models
#20I've yet to see data this is beating a SVM on a 4-character shingle approach (which also doesn't require tons of data to train).
It beats it easily. See the paper and citations for comparison