Paving the way for human-level sentence corrections
tech.grammarly.com
Paving the way for human-level sentence corrections
1–10 of 23 posts
Re: Paving the way for human-level sentence corrections
#2This was an interesting read for someone unacquainted with the field–it appears to be very difficult to fix "awkwardness" in sentences; none of the methods were able to reduce it significantly. It looks to me that awkwardness is more based on common usage than on actual grammar, perhaps this could be improved with a solution similar to Google Translate's, which looks at real world usage instead of syntax?
Re: Paving the way for human-level sentence corrections
#3Fluency is nice, but semantics matter the most.
Re: Paving the way for human-level sentence corrections
#4This was an interesting read for someone unacquainted with the field–it appears to be very difficult to fix "awkwardness" in sentences; none of the methods were able to reduce it significantly. It looks to me that awkwardness is more based on common usage than on actual grammar, perhaps this could be improved with a solution similar to Google Translate's, which looks at real world usage instead of syntax?
Real world usage would have to be curated though, awkward sentence constructions or word choices do happen in real world usage. Or as the article shows, there can be multiple, very different ways of fixing awkwardness. I'm not sure what it would look like to find a solution that's "fitted" to several of these.
Re: Paving the way for human-level sentence corrections
#5Hmm I'm sensing a bit of garbage-in garbage-out here. For starters their original sentences contain unlikely typos instead of homonyms which would be much more commons. (it complicates the learning as well I'm sure since some changes were made to correct similarly spelled terms which could really change a sentence's meaning once applied) Second, the human corrections aren't that good. We really need to stop creating data sets using anonymous exploited labor that is paid pennies. (they did screen the Amazon Turk users, but if you live in America or work at a university, is there really a shortage of fluent English speakers around you?) Overall I'd say the fluency-editing approach shows promise and would be a boon to ESL-learners, but the training data needs to be improved.
Re: Paving the way for human-level sentence corrections
#6I have a feeling this is one of those places where ML will not be useful until we have strong AI.
Certain grammatical errors are impossible to fix unless you understand the overall meaning of the text. Sometimes this meaning is embedded over many paragraphs. Errors involving incorrect word usage are unsolvable when words have more than one meaning and you don't comprehend the subject at hand.
Re: Paving the way for human-level sentence corrections
#7This is a great project -- in Phase One, the algorithm will correct sentences written by people who didn't learn basic literacy in school and who subsequently endeavor to avoid reading or writing any text, preferring video. In Phase Two, the algorithm will do away with the poorly written source and create something entirely on its own. Based on my sampling of contemporary human-crafted sentences, Phase Two will take place just in time.
Apropos, my all-time favorite malapropism took place 50 years ago when I was a teenage TV repairman. I visited a household, spied a record turntable, and asked, "Is that a stereo turntable?" "No," replied the customer, "It's monorail."
I was able to avoid blurting out, "I think you mean monaural, yes?" -- for three reasons. One, it's regarded as bad form to correct the grammar of customers, who are always right. Two, technically, the turntable was in fact monorail (i.e. able to follow only one recorded track). Three, I was too busy trying not to laugh.
Re: Paving the way for human-level sentence corrections
#8The problem that the authors are trying to tackle is an interesting and difficult one.
I have noticed that when dealing with natural language using artificial intelligence / machine learning techniques, the work being done by computer scientists very often would have greatly benefitted from collaboration with a linguist or other sort of language expert, especially in the design phase of an experiment. This work is a good example of what I mean.
People trained in CS or similar precise fields develop, over time, a tendency of thinking in terms of "getting the right result" (I say this as one of these people). When dealing with natural language, however, sometimes there simply is no single correct result.
Consider the topic of fluency that the authors work on: is there a rigorous, objective definition of "fluent"? The answer, as any linguist would tell you, is "no". There are idiomatic expressions, grammatical structures, contractions, slang, and so on that vary from city to city within a country, let alone globally. What may sound "fluent" to one native speaker of a language may sounds strange to another. It is impossible to objectively generally evaluate "fluency". In particular, any practicing linguist will be able to give examples, likely off the top of their head, of English sentences that would be rated as "fluent" by someone from one geographical area and "awkward" by someone from another.
Furthermore, using Mechanical Turk to find humans to rate the fluency of a particular sentence makes for an unclean dataset and evaluation benchmark. The linked post says that, in the end, 50 people found via Mechanical Turk rated sentences for fluency; since any one language is used significantly differently around the globe, there will be an unpredictable range of fluency ratings for at least some sentences across just 50 people around the world. Choosing a different 50 people to rate the same sentences would most likely result in different fluency ratings.
I do not mean to detract from the authors' work; this is a difficult problem to tackle, with no clear path to a general solution. However, I am forced to wonder why the authors, who, based on their biographies linked to in the article, seem to have a range of experience, did not comment on the considerations I've mentioned here.
Re: Paving the way for human-level sentence corrections
#9I have a feeling this is one of those places where ML will not be useful until we have strong AI. Certain grammatical errors are impossible to fix unless you understand the overall meaning of the text. Sometimes this meaning is embedded over many paragraphs. Errors involving incorrect word usage are unsolvable when words have more than one meaning and you don't comprehend the subject at hand.
You don't think we can "fake it" in the the vast statistical majority of cases simply by relying on a corpus containing nearly the same cases?
We can already "understand the meaning" in a latent space well enough to do machine translation between language pairs the model wasn't trained on, or do additions and subtractions in the latent space of word to vec to suggest they have picked up some semantic meaning from the text.
I don't think this is a problem that requires Strong AI in the vast majority of cases, just very large well groomed corpa and clever engineers.
Re: Paving the way for human-level sentence corrections
#10s/are comprised of/are composed of/