"I gotta take shit" -> "I gotta take the shit" Sorry to be airing out my personal business and everything but... everybody poops!
Deep Text Correcter
61–70 of 71 posts
Re: Deep Text Correcter
#62The "correcter" is a worthy effort, and it needs to start somewhere. It shows the magnitude of the task considering that missing articles are not the most crucial grammatical issues in on-line discourse. The meaning of a phrase is usually comprehensible with or without the article, and native speakers can easily overlook this kind of error made by non-native speakers.
OTOH more troublesome to readers are common errors such as misuse of "its" vs. "it's", "to" vs. "too", and "their", "there" and "they're". These mistakes are quite prevalent among native-speaking writers so more ubiquitous than the missing article problem.
The "correcter" didn't correct the latter class of errors. Understandably this would be a much harder goal to accomplish given the highly contextual nature of grammatically correct word choices.
It prompts a question about how well the data-driven approach can handle the problem. Obviously that's what the research is trying to answer. It sure seems to point to something fairly easy for a human to do that's near or at the limit of what we can get a computer to do.
Re: Deep Text Correcter
#63>> Thus far, these perturbations have been limited to: + the subtraction of articles (a, an, the) + the subtraction of the second part of a verb contraction (e.g. “‘ve”, “‘ll”, “‘s”, “‘m”) + the replacement of a few common homophones with one of their counterparts (e.g. replacing “their” with “there”, “then” with “than”) Oooh, that's _very_ tricky what they're trying to do there. "Perturbations" that cause grammatica…
So what? Who says the training data has to be perfect? Your comment reminds me of an essay by Peter Norvig. Where he showed that a simple statistical model could tell the difference between a nonsense sentence that was grammatically correct and one that wasn't, but gave them both very low probability of occuring naturally. Similarly, the machine may learn that "Give him the flowers" has higher probability than "give…
You're talking about the grammatical, but nonsensical sentence "colorless
green ideas sleep furiously" proposed by Chomsky, along with its ungrammatical
counterpart, "Furiously sleep ideas green colorless." [1] Chomsky famously
said that one of the two sentences is ungrammatical, but both would be assigned a low
probability by a statistical model trained on English corpora. The
one who showed that the ungrammatical sentence was (much) less likely than
the grammatical one was Fernando Pereira, not Peter Norvig. Peter Norvig has
argued against Chomsky in a different instance, and cited Pereira during that
(one-sided) exchange, if memory serves.
This is completely unrelated to what I'm saying, of course. The fact of the matter is that you can't just swap some words out of a sentence and call it "ungrammatical". You can call it a "perturbed" sentence. You can call it a "sentence with some words dropped". You can call it Daisy Duck for all I care. But it's not an ungrammatical sentence, because that sort of thing has a very precise definition, which I gave above.
_______________
[1] The sentence is famous enough to have a wikipedia page all its own: https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_fu...
Re: Deep Text Correcter
#64Earlier quoted context omitted.
Dropping `in the fridge` works. "Alex went to kitchen to store the milk" corrects to "Alex went to the kitchen to store the milk"
On the other hand, it never removes unnecessary articles, demonstrating one of the deficiencies of the training set.
Couldn't that be rectified with a Parts of Speech tagger? Or am I overlooking something?
Re: Deep Text Correcter
#65>> Thus far, these perturbations have been limited to: + the subtraction of articles (a, an, the) + the subtraction of the second part of a verb contraction (e.g. “‘ve”, “‘ll”, “‘s”, “‘m”) + the replacement of a few common homophones with one of their counterparts (e.g. replacing “their” with “there”, “then” with “than”) Oooh, that's _very_ tricky what they're trying to do there. "Perturbations" that cause grammatica…
https://encrypted.google.com/search?q=%22give%20him%20flower... – if your data set is good enough, you won't be suggesting insertion there.
Maybe calling the perturbed (and model-filtered!) sentence "ungrammatical" is too strong a statement, but that doesn't stop the system from being useful. We don't have a perfect model of NL grammar, but nor do we have complete dictionaries of any NL lexicons – we still rely on spell checkers because they have fairly acceptable false positive rates. Spell checkers also do these perturbations: Levenshtein-edits. And some times they generate sequences that the model (dictionary) will remove because it's already in there with some frequency and they generate a true negative, some times they generate sequences that were simply missing from their data set and they generate a false positive, and some times they generate a sequence that is neither in their model nor in the language and you get a true positive. The same general principle applies to grammar checkers (but they of course are much harder to make with an acceptable false positive rate, and it's much more difficult to generate confusion sets).
Re: Deep Text Correcter
#66Interesting idea! I think this is analogous to the idea of a de-noising autoencoder in computer vision. Here, instead of introducing Gaussian noise at the pixel level and using a CNN, you're introducing grammatical "noise" at the world level and using an LSTM. I think that general framework applies to many different domains. For example, we trained a denoising sequence autoencoder on HealthKit data (sequences of step…
It seems like it would be challenging to get the corruptor to generate examples that are of the same Kind that humans make, while still being "productive" (in the linguistic sense, ie. not just overfitting on examples from a corpus of low quality text).
It's easy enough to just drop random words or run Levenshtein-edits on single words to create wrong-in-this-context makes (then/than), but grammar errors include much more than can be covered by that method, and the method will generate many errors that are of a kind never made by humans. And if you restrict your method to things already seen in a corpus, it's easy to overfit and miss out on a whole lot of good stuff.
Re: Deep Text Correcter
#67Nice work! I was playing with exactly this idea for some time. Potentially it could be way bigger than simple grammatical corrections.
My list of things to try, in addition to what you've already done:
- replacing named entities with metadata-annotated tokens;
- dropping random words, not just articles;
- replacing random words with rarer synonyms;
- annotate with POS tags from some external parser;
- run syntax corrector before feeding sentences in grammatical model;
I think this problem is easier that it appears on the surface. Generated deformation does not have to be a perfect replica of typical human errors. It just have to be sufficiently diverse.
Also, I think seq2seq module is getting deprecated, as it doesn't do dynamic rollouts.
Re: Deep Text Correcter
#68isn't a ngram bayesian model sufficient here?
also, isn't the test data linear dependent from the training set here so creating skewed performance measurement?
Re: Deep Text Correcter
#69Earlier quoted context omitted.
On the other hand, it never removes unnecessary articles, demonstrating one of the deficiencies of the training set.
Couldn't that be rectified with a Parts of Speech tagger? Or am I overlooking something?
The training data never says to remove any articles, so a PoS tagger won't help.
Re: Deep Text Correcter
#70>> Thus far, these perturbations have been limited to: + the subtraction of articles (a, an, the) + the subtraction of the second part of a verb contraction (e.g. “‘ve”, “‘ll”, “‘s”, “‘m”) + the replacement of a few common homophones with one of their counterparts (e.g. replacing “their” with “there”, “then” with “than”) Oooh, that's _very_ tricky what they're trying to do there. "Perturbations" that cause grammatica…
https://encrypted.google.com/search?q=%22give%20him%20flower... – if your data set is good enough, you won't be suggesting insertion there. Maybe calling the perturbed (and model-filtered!) sentence "ungrammatical" is too strong a statement, but that doesn't stop the system from being useful. We don't have a perfect model of NL grammar, but nor do we have complete dictionaries of any NL lexicons – we still rely on sp…
>> – if your data set is good enough, you won't be suggesting insertion there.
Generally, I doubt you should ever suggest any corrections to the use of articles. This tends to be colloquial, or even personal and there's no point trying to force one style on your users. Unless you like pissing them off (MS Word sure does).
As to spellcheckers- I think that goes the other way, doesn't it? You apply some edits to an out-of-vocabulary word you found, to see whether you can arrive at an in-vocabulary word to suggest as a correction.
In any case, I'm not disagreeing with anything you say. I'm just saying that "grammatical" and "ungrammatical" have strict definitions and you can't just throw them around like skittles.