Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: A tool for writing English that checks “popularity” of used sentences?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 51 posts

Ask HN: A tool for writing English that checks “popularity” of used sentences?

#1

Post by twa927 »

As a non-native English speaker I find that the best way to check grammar is to google whole parts of sentences (in apostrophes - exact match). It's because there are multiple exceptions to language rules and some wording just can feel "not right" despite being correct.

Is there a tool that does something like this automatically?

I thought about writing such tool by myself, but it seems there are no good-quality, free search engine APIs that allow many calls. Or, maybe there are some open APIs to book dumps or something similar?

Re: Ask HN: A tool for writing English that checks “popularity” of used sentences?

#2

Post by barryhunter »

There are quite a few Ngram datasets available https://www.google.com/search?q=download+n-gram+dataset

... these are almost certainly used in many spelling and grammar checkers. (To help with where the same spelled word is used in different context)

http://www.aclweb.org/anthology/W12-0304

Re: Ask HN: A tool for writing English that checks “popularity” of used sentences?

#3

Post by aytekin »

I wonder if there is a tool like this:

1. You enter a sentence

2. It gives out 5 different ways to say the exact same thing.

Such a tool not only would help ESL people but also it would help native speakers find more relaxed or formal versions of a sentence.

Re: Ask HN: A tool for writing English that checks “popularity” of used sentences?

#4

Post by Xeoncross »

You could probably use some of the Ngrams datasets to figure this out. Parse some books from https://www.gutenberg.org/ or use the google ngrams corpus. Pay attention to the year(s) which you wish to model english from - grammar and form keep changing!

Re: Ask HN: A tool for writing English that checks “popularity” of used sentences?

#5

Post by matt4077 »

aytekin wrote: post #3

I wonder if there is a tool like this: 1. You enter a sentence 2. It gives out 5 different ways to say the exact same thing. Such a tool not only would help ESL people but also it would help native speakers find more relaxed or formal versions of a sentence.

It's a pretty hard problem to solve in for the general case. It's actually quite rare for sentences to not be unique. Example: https://www.google.de/search?rls=en&q=%22It+gives+out+5+diff...

Re: Ask HN: A tool for writing English that checks “popularity” of used sentences?

#7

Post by twa927 »

barryhunter wrote: post #2

There are quite a few Ngram datasets available https://www.google.com/search?q=download+n-gram+dataset ... these are almost certainly used in many spelling and grammar checkers. (To help with where the same spelled word is used in different context) http://www.aclweb.org/anthology/W12-0304

Yes, I remember trying to use Google Books Ngram Dataset [1], but it was too tedious for me to setup and maintain a server with the data for a purpose of a quick-and-dirty tool (that's why I asked for a ready API). Still, using it is probably a nice idea for a more ambitious side project or even a startup.

EDIT. Actually I would happily pay for a tool that implements the idea. Grammarly has paid plans but $30/month is too steep (for my types of usages), and the types of grammar checks it performs is not exactly what I need (which is what real people in real situations use).

[1] http://storage.googleapis.com/books/ngrams/books/datasetsv2....

Re: Ask HN: A tool for writing English that checks “popularity” of used sentences?

#8

Post by twa927 »

matt4077 wrote: post #5
aytekin wrote: post #3

I wonder if there is a tool like this: 1. You enter a sentence 2. It gives out 5 different ways to say the exact same thing. Such a tool not only would help ESL people but also it would help native speakers find more relaxed or formal versions of a sentence.

It's a pretty hard problem to solve in for the general case. It's actually quite rare for sentences to not be unique. Example: https://www.google.de/search?rls=en&q=%22It+gives+out+5+diff...

Yet both "It gives out" and "different ways to say the exact same thing" give many thousands of results. So "5" should be recognized as a template variable or the meaning should be combined from popular fragments.

Re: Ask HN: A tool for writing English that checks “popularity” of used sentences?

#9

Post by infinitone »

aytekin wrote: post #3

I wonder if there is a tool like this: 1. You enter a sentence 2. It gives out 5 different ways to say the exact same thing. Such a tool not only would help ESL people but also it would help native speakers find more relaxed or formal versions of a sentence.

We're building a tool that does something similar but for email. Currently we're targeting cold sales emails- the idea is, you enter a recipient's email and we aggregate data about them and surface relevant, personable sentences that you can use in the email. You'll also be able to change the tone of these sentences (funny, professional, casual, etc.)

Learn more: http://emailfox.co

Re: Ask HN: A tool for writing English that checks “popularity” of used sentences?

#10

Post by mrob »

barryhunter wrote: post #2

There are quite a few Ngram datasets available https://www.google.com/search?q=download+n-gram+dataset ... these are almost certainly used in many spelling and grammar checkers. (To help with where the same spelled word is used in different context) http://www.aclweb.org/anthology/W12-0304

LanguageTool has limited support for using Google's n-gram data to find spelling errors. It only uses 3-grams, and only for a list of commonly confused words. I'm not aware of any Free Software that does better.

http://wiki.languagetool.org/finding-errors-using-n-gram-dat...

Post reply on HN