Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep Text Correcter

atpaino.com

Post reply on HN
Print view View original

51–60 of 71 posts

Re: Deep Text Correcter

#51

Post by YeGoblynQueenne »

>> Thus far, these perturbations have been limited to:

  + the subtraction of articles (a, an, the)
  + the subtraction of the second part of a verb contraction (e.g. “‘ve”, “‘ll”, “‘s”, “‘m”)
  + the replacement of a few common homophones with one of their counterparts (e.g. replacing “their” with “there”, “then” with “than”)
Oooh, that's _very_ tricky what they're trying to do there.

"Perturbations" that cause grammatical sentences to become ungrammatical are _very_ hard to create, for the absolutely practical reason that the only way to know whether a sentence is ungrammatical is to check that a grammar rejects it. And, for English (and generally natural languages) we have no (complete) such grammars. In fact, that's the whole point of language modelling- everyone's trying to "model" (i.e. approximate, i.e. guess at) the structure of English (etc)... because nobody has a complete grammar of it!

Dropping a few bits off sentences may sound like a reasonable alternative (an approximation of an ungrammaticalising perturbation) but, unfortunately, it's really, really not that simple.

For instance, take the removal of articles: consider the sentence: "Give him the flowers". Drop the "the". Now you have "Give him flowers". Which is perfectly correct and entirely plausible, conversational, everyday English.

In fact, dropping words is de rigeur in language modelling, either to generate skip-grams for training, or to clean up a corpus by removing "stop words" (uninformative words like the the and and's) or generally, cruft.

For this reason you'll notice that the NUCLE corpus used in the CoNLL-2014 error correction task mentioned in the OP is not auto-generated, and instead consists of student essays corrected by professors of English.

tl;dr: You can't rely on generating ungrammaticality unless you can generate grammaticallity.

Re: Deep Text Correcter

#52

Post by YeGoblynQueenne »

jmiserez wrote: post #7

Interesting idea. I went ahead and tested: > Alex went to the kitchen to store the milk in the fridge. Corrected: > Alex went to the kitchen to the store the milk in the fridge. Gathering a large, high quality dataset from the internet is probably not so easy. A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself). Movie dialogues don…

>> But a tool that could understand my intent and then propose restructured or similar sentences and words that convey the same meaning could be a true killer application.

Yeah but that's hard though. Who knows what is your intent when you're generating an utterance, written or spoken? Sometimes you yourself may even read what you wrote (or hear what you said) and wonder what you meant.

Not to mention what an absolute nightmare it would be, trying to compile data on the linguistic intent behind utterances! How do you even start to collect that? Ask people to say things, then ask them what they meant... but what did they mean when they explain what they meant in the first instance?

The worse thing is that the very notion of what is grammatical changes with context. For instance, to go back to dropping the "the"'s: imagine you read the phrase "eat soup with spoon". Is that an ungrammatical form of "eat the soop with the spoon", or is it an instruction, perhaps something you'd find in a soup-eating manual, which therefore is perfectly valid as a terse form of English?

What you intend an utterance to mean affects whether it is grammatical and the grammaticality of the utterance affects its meaning. Nice, eh?

Re: Deep Text Correcter

#54

Post by nmstoker »

YeGoblynQueenne wrote: post #52
jmiserez wrote: post #7

Interesting idea. I went ahead and tested: > Alex went to the kitchen to store the milk in the fridge. Corrected: > Alex went to the kitchen to the store the milk in the fridge. Gathering a large, high quality dataset from the internet is probably not so easy. A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself). Movie dialogues don…

>> But a tool that could understand my intent and then propose restructured or similar sentences and words that convey the same meaning could be a true killer application. Yeah but that's hard though. Who knows what is your intent when you're generating an utterance, written or spoken? Sometimes you yourself may even read what you wrote (or hear what you said) and wonder what you meant. Not to mention what an absolut…

You make a great point.

Presumably biasing the training sets to those built from corrected sources would help, but what about using n-grams to throw out poor examples of ungrammatical sentences? It wouldn't be perfect, but some kind of score based on a much lower prevlence of n-grams from the generated sentence vs the original might indicate acceptable cases and those where they were fairly similar might indicate the generated sentence could in fact be valid (and thus should be discarded)

Re: Deep Text Correcter

#55

Post by [deleted] »

YeGoblynQueenne wrote: post #52
jmiserez wrote: post #7

Interesting idea. I went ahead and tested: > Alex went to the kitchen to store the milk in the fridge. Corrected: > Alex went to the kitchen to the store the milk in the fridge. Gathering a large, high quality dataset from the internet is probably not so easy. A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself). Movie dialogues don…

>> But a tool that could understand my intent and then propose restructured or similar sentences and words that convey the same meaning could be a true killer application. Yeah but that's hard though. Who knows what is your intent when you're generating an utterance, written or spoken? Sometimes you yourself may even read what you wrote (or hear what you said) and wonder what you meant. Not to mention what an absolut…

[deleted]

Re: Deep Text Correcter

#56

Post by Houshalter »

YeGoblynQueenne wrote: post #51

>> Thus far, these perturbations have been limited to: + the subtraction of articles (a, an, the) + the subtraction of the second part of a verb contraction (e.g. “‘ve”, “‘ll”, “‘s”, “‘m”) + the replacement of a few common homophones with one of their counterparts (e.g. replacing “their” with “there”, “then” with “than”) Oooh, that's _very_ tricky what they're trying to do there. "Perturbations" that cause grammatica…

So what? Who says the training data has to be perfect? Your comment reminds me of an essay by Peter Norvig. Where he showed that a simple statistical model could tell the difference between a nonsense sentence that was grammatically correct and one that wasn't, but gave them both very low probability of occuring naturally.

Similarly, the machine may learn that "Give him the flowers" has higher probability than "give him flowers". Or it may learn that both are possible sentences and not be able to correct it. Which is also OK, we can't expect any system to be perfect.

Re: Deep Text Correcter

#57

Post by tyingq »

johnhenry wrote: post #44
raverbashing wrote: post #39

While there are limited errors it can officially correct, I tried a few phrases: Didn't fix misuse of its: "The tool worked on it's own power" "He should of gone yesterday" gets corrected to "the He should of gone yesterday" "To who does this belong?" doesn't get corrected "A Apple a day keeps the doctor away" doesn't change

I tried these, which stem from the given example: "I'm going to store" gets corrected to "I'm going to the store" (looks good) "I'm going to store food" gets corrected to "I'm going to the store store" (I'll forgive this due to lack of a prepositional phrase) "I'm going to store food in the closet" gets corrected to "I'm going to the store food in the closet" (yeah, this is wrong) It seems like the AI is good at fixi…

It does seem to pick up on the specific cases where adding "the" isn't right, like:

Alex went to school, Alex went to bed, Alex went to prison

Re: Deep Text Correcter

#58

Post by atpaino »

brandonb wrote: post #25

Interesting idea! I think this is analogous to the idea of a de-noising autoencoder in computer vision. Here, instead of introducing Gaussian noise at the pixel level and using a CNN, you're introducing grammatical "noise" at the world level and using an LSTM. I think that general framework applies to many different domains. For example, we trained a denoising sequence autoencoder on HealthKit data (sequences of step…

I like the analogy to de-noising autoencoders; that's a good way of thinking about this.

> In the limit, you might even have a neural network generate increasingly harder types of grammatical corruptions, with the goal of "fooling" the corrector network.

Very interesting. I wonder how many constraints would need to be added to the corruptor model to ensure the corrupted sentence retains the same meaning as the original. Somewhat related to that, I've thought that a more basic curriculum learning setup could be deployed quite effectively here, and am hoping to try that out soon.

Re: Deep Text Correcter

#59

Post by vacri »

> "Kvothe went to market"

This is not a grammatically incorrect sentence; it depends on context. Products are take to an abstract concept of 'market', for example.

Re: Deep Text Correcter

#60

Post by lern_too_spel »

aembleton wrote: post #17
jmiserez wrote: post #7

Interesting idea. I went ahead and tested: > Alex went to the kitchen to store the milk in the fridge. Corrected: > Alex went to the kitchen to the store the milk in the fridge. Gathering a large, high quality dataset from the internet is probably not so easy. A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself). Movie dialogues don…

Dropping `in the fridge` works. "Alex went to kitchen to store the milk" corrects to "Alex went to the kitchen to store the milk"

On the other hand, it never removes unnecessary articles, demonstrating one of the deficiencies of the training set.
Post reply on HN