21–30 of 71 posts
Original: The complex houses married and single soldiers and their families.
Deep Text Corrector: The complex houses married and a single soldiers and their families.
OT: does anyone know of a more substantial list of garden path sentences that people use in testing NLP software?
Interesting idea. I went ahead and tested: > Alex went to the kitchen to store the milk in the fridge. Corrected: > Alex went to the kitchen to the store the milk in the fridge. Gathering a large, high quality dataset from the internet is probably not so easy. A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself). Movie dialogues don…
> A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself). Corrected: > A lot of the content on the HN/Reddit/forums is of the quality grammatically and non-native written by written English speakers (such myself). as UNK Yeah. It's got a long way to go. No idea where "as UNK" came from.
Earlier quoted context omitted.
Guess who's got that massive amount of data nowadays ? Google, facebook and co. So good luck if you want to work on AI stuff...
And Project Gutenberg: https://www.gutenberg.org/ The language in many of those books may be a bit archaic, though. See also Distributed Proofreaders: http://www.pgdp.net/c/
I think that general framework applies to many different domains. For example, we trained a denoising sequence autoencoder on HealthKit data (sequences of step counts and heart rate measurements) in order to predict whether somebody is likely to have diabetes, high blood pressure, or a heart rhythm disorder based on wearable data. I've also seen similar ideas applied to EMR data (similar to word2vec). It's worth reading "Semi-Supervised Sequence Learning", where they use a non-denoising sequence autoencoder as a pretraining step, and compare a couple of different techniques: https://papers.nips.cc/paper/5949-semi-supervised-sequence-l...
Toward the end, you start thinking about introducing different types of grammatical errors, like subject-verb disagreement. I think that's a good way to think about it. In the limit, you might even have a neural network generate increasingly harder types of grammatical corruptions, with the goal of "fooling" the corrector network. As the the corruptor network and corrector network compete with each other, you might end up with something like a generative adversarial network: https://arxiv.org/abs/1701.00160
Earlier quoted context omitted.
(Comment removed.)
Well, I think the dictionary disagrees. http://www.dictionary.com/browse/corrector I'm going to assume that it was meant to be ironic (or errorful).
For example:
> Do you know where I been
'corrects' to:
> Do you know where I 's been
This is really cool. If you're looking for more datasets to train your model, here are a few relevant ones: - https://archive.org/details/stackexchange - http://trec.nist.gov/data/qamain.html - http://opus.lingfil.uu.se/OpenSubtitles2016.php - http://corpus.byu.edu/full-text/wikipedia.asp OR https://en.wikipedia.org/wiki/Wikipedia:Database_download#En... - http://opus.lingfil.uu.se/ I'd love to see how good your mode…
Thanks for the links! I'll have to try out some of these. The data is definitely the limiting factor at this point.
Is there any reason why that huge corpus would not be useful?
Interesting idea. I went ahead and tested: > Alex went to the kitchen to store the milk in the fridge. Corrected: > Alex went to the kitchen to the store the milk in the fridge. Gathering a large, high quality dataset from the internet is probably not so easy. A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself). Movie dialogues don…
Maybe it's also to do with how they generate the training data. Author did say removal of articles was one thing they used to generate the 'incorrect' sentence in the training data. If the film data uses 'the store' much more than 'store', you can imagine how the algo could get biased to thinking store is always preceded with 'the'.