Live data from Hacker News

A spellchecker used to be a major feat of software engineering (2008)

prog21.dadgum.com

91–100 of 154 posts

Re: A spellchecker used to be a major feat of software engineering (2008)

#91

    Spell chequer by Martha Snow


    Eye halve a spelling chequer
    It came with my pea sea
    It plainly marques four my revue
    Miss steaks eye kin knot sea.

    Eye strike a quay and type a word
    And weight four it two say
    Weather eye am wrong oar write
    It shows me strait a weigh.

    As soon as a mist ache is maid
    It nose bee fore two long
    And eye can put the error rite
    It's rare lea ever wrong.

    Eye have run this poem threw it
    I am shore your pleased two no
    It's letter perfect awl the weigh
    My chequer tolled me sew.

Re: A spellchecker used to be a major feat of software engineering (2008)

#92

...and yet the first thing I switch off in every tool is the spellchecker, because they get utterly confused with the mix of English, German and my local dialect that I'm using when communicating with different people. I'd say doing a spellchecker "right" is apparently still a major feat of software engineering ;)

I wish it just struggled with the mix of multiple languages, it can't even handle my native one. A plural compound word in translative case with a clitic? I can only dream about it, it can't even do all the cases reliably.

Re: A spellchecker used to be a major feat of software engineering (2008)

#93
post #3

Google's spell checker still is a major feat of software engineering ;-) It must handle every language, handle new words constantly being coined, have low enough latency and high enough throughput to run it on every web search. (Well, caching probably solves 2/3 of traffic.)

I find googles spell checking to be the best in the world. Google search is where I go when local spell checkers failed.

I dunno, really. It is much better than Word on English, but it is laughable when I jumble several languages on a single document, and doesn't handle some laguages that relies on code-switching. At least Word knows its boundaries.

Re: A spellchecker used to be a major feat of software engineering (2008)

#94
post #3

Google's spell checker still is a major feat of software engineering ;-) It must handle every language, handle new words constantly being coined, have low enough latency and high enough throughput to run it on every web search. (Well, caching probably solves 2/3 of traffic.)

While it's not bad, it's not that great a universal tool either. My native language makes use of a lot of compound words and Google's spell checker often gets confused when I combine words according to the standard grammar. I can see and understand the technical limitations, but tools like Microsoft Word seem to do a much better job than Google's spell checker, even in things like Google Docs. Google search will ofte…

This exactly. I've often wondered if the default spell checkers that come with phones nowadays are slowly changing the language itself. If Google doesn't fix these issues in the nearish future, an entire generation will grow up with a device that tells them the spelling they learnt in school is wrong.

And with so many people just accepting Google's anti-compound spelling as correct, many people will be mostly exposed to a version of their language with more spaces, and it'll start to "feel" right to them.

Re: A spellchecker used to be a major feat of software engineering (2008)

#95
post #3

Google's spell checker still is a major feat of software engineering ;-) It must handle every language, handle new words constantly being coined, have low enough latency and high enough throughput to run it on every web search. (Well, caching probably solves 2/3 of traffic.)

I feel it's more statistics based than true spellchecking? At least I often feel that it wants to rewrite my sentences to something completely different because what I'm searching/writing is uncommon but similar to a more common word.

Re: A spellchecker used to be a major feat of software engineering (2008)

#96

I don't want to brag, but it wasn't that hard to write a spellchecker in 256KB in 1984. I was in primary school (age 11) and we had a Microbee at home with 64KB of RAM and a copy of Turbo Pascal. My sister is/was dyslexic so with a bit of help from my older brother (who would have been in first year of computer science at the time) I wrote a spellchecker for her. It think it might have been a Christmas present, so I…

A spell checker isn't something in a binary state of working or not working. Its something that has the almost impossible task of working out what the user wanted and not what they asked for. Maybe most of your problems are a single letter mistake or a keyboard slip up but where good spell checkers shine is they know what you want even when you are miles off. I find googles spell checking to be exceptional at underst…

There are approaches that take into account how words sound like e.g. metaphone

Re: A spellchecker used to be a major feat of software engineering (2008)

#97

...and yet the first thing I switch off in every tool is the spellchecker, because they get utterly confused with the mix of English, German and my local dialect that I'm using when communicating with different people. I'd say doing a spellchecker "right" is apparently still a major feat of software engineering ;)

Yeah, to add to it a lot of Germans will use American English but some British English and my spellchecker can't handle the mixture. Off topic but despide how difficult German has been to learn I've found spelling in German to be far more predictable and straightforward than my native language English.

Re: A spellchecker used to be a major feat of software engineering (2008)

#98
I get the point of the article that today's improved hardware allows things that weren't possible before.

But neither spell checking nor its more advanced siblings spelling correction or automatic spelling correction is solved. Here is an example:

"I want to by Apple"

"I want to be Apple"

"I want to buy Apple"

Because "by" is a genuine English word, a spellchecker based on a simple hashtable will not detect the error, which obviously exists. We need to look at the context of the whole sentence or even document. Today we can use deep learning and word embedding to solve/improve that problem.

A simple spelling corrector that is based on edit distance would always prefer "be" over "buy" because "be" is the more frequent word of both.

So the risk of all the easy accessible libraries for a certain task is, that you think the problem is solved, but you overlook many problems because the topic is much more diverse and deep than you would think at the first glance

Another point is that while hardware improved, also the expectations have risen. We expect milliseconds latency instead of seconds, we often require the software to work in a multi-tenant architecture with thousands or millions of concurrent users.

So even for seemingly simple and straightforward tasks, and despite all the exciting advances in hardware, always new challenges present themselves.

Re: A spellchecker used to be a major feat of software engineering (2008)

#99
Maybe it should've been a feat left unaccomplished. Far too often do we have software trying to carry out the impossible task of guessing what the user wants without asking. I don't want the computer to tell me what I want, I just want it to do what I tell it to do.
Post reply on HN