31–40 of 71 posts
Earlier quoted context omitted.
Thanks for the links! I'll have to try out some of these. The data is definitely the limiting factor at this point.
Have you consider project Gutenberg? Is there any reason why that huge corpus would not be useful? https://www.gutenberg.org/
Interesting idea. I went ahead and tested: > Alex went to the kitchen to store the milk in the fridge. Corrected: > Alex went to the kitchen to the store the milk in the fridge. Gathering a large, high quality dataset from the internet is probably not so easy. A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself). Movie dialogues don…
> The player chose to go to the good team rather than the sucky one.
> The player chose to go to the good team than rather the sucky one.
And it seems to like duplicating adjectives:
> Being worse off than even worse off makes one feel sad.
> Being worse worse than even worse worse makes one feel sad.
Earlier quoted context omitted.
Have you consider project Gutenberg? Is there any reason why that huge corpus would not be useful? https://www.gutenberg.org/
I think the issue is you need parallel corpuses of (sentences with grammatically errors) -> (same sentence but correct). Gutenberg has a lot of only the latter I think.
Earlier quoted context omitted.
Well, I think the dictionary disagrees. http://www.dictionary.com/browse/corrector I'm going to assume that it was meant to be ironic (or errorful).
The demo itself agrees. It changes the word correcter, but not the word corrector :)
Genius. ;)
> Unfortunately, I am not aware of any publicly available dataset of (mostly) grammatically correct English. How about books?
Guess who's got that massive amount of data nowadays ? Google, facebook and co. So good luck if you want to work on AI stuff...
I recommend http://opus.lingfil.uu.se/ as a starting point to find lots of sources of quality, multilingual data.
And if all you want is sheer quantity, at rather the expense of quality, then there's a several terabyte Web crawl at http://commoncrawl.org/.
> Unfortunately, I am not aware of any publicly available dataset of (mostly) grammatically correct English. How about books?
Online forums are a disaster (HN is not too bad, but there are a lot of natives and non-natives fumbling some things)
Interesting idea. I went ahead and tested: > Alex went to the kitchen to store the milk in the fridge. Corrected: > Alex went to the kitchen to the store the milk in the fridge. Gathering a large, high quality dataset from the internet is probably not so easy. A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself). Movie dialogues don…
> A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself). Corrected: > A lot of the content on the HN/Reddit/forums is of the quality grammatically and non-native written by written English speakers (such myself). as UNK Yeah. It's got a long way to go. No idea where "as UNK" came from.
coincidentally, the op corrected this as: It's like the equivalent of NaN
Didn't fix misuse of its: "The tool worked on it's own power"
"He should of gone yesterday" gets corrected to "the He should of gone yesterday"
"To who does this belong?" doesn't get corrected
"A Apple a day keeps the doctor away" doesn't change