Live data from Hacker News

A spellchecker used to be a major feat of software engineering (2008)

prog21.dadgum.com

11–20 of 154 posts

Re: A spellchecker used to be a major feat of software engineering (2008)

#11
post #6

If you haven't seen it already you should check out Peter Norvig's 20-odd line toy spell checker, written over the course of a flight. https://norvig.com/spell-correct.html

Here's D's spell checker, tests included!

Edit: forgot the link https://github.com/dlang/dmd/blob/master/src/dmd/root/spelle...

Re: A spellchecker used to be a major feat of software engineering (2008)

#13

I especially like the spellchecker added to the D compiler. The neato feature is the "dictionary" is the part of the symbol table that is in scope. In my usage it guesses right about 50-75% of the time. I've been considering adding one to my text editor. I found out I'm not as good a speller as I thought I was before spellcheckers :-/

> I've been considering adding one to my text editor. I found out I'm not as good a speller as I thought I was before spellcheckers :-/

We all make mistakes and it’s okay, there is no shame in it! Let the computer fill in the gaps for you so you can use your brain for other things :) I shamelessly use a spell checker in my IDE and it has been a very positive experience, highly recommend!

Re: A spellchecker used to be a major feat of software engineering (2008)

#14
Arguably detecting typographical (or transcription) errors is still non-trivial today since a) edit distance is NP complete and b) selecting the correct spelling often depends on grammar as well as semantic context.

For example, consider the erroneous phrase "he was put through the ringer." Although "ringer" matches a spelling in the dictionary, it doesn't make sense semantically (a "ringer" being a device that rings bells, a near-duplicate of something else, etc.) and the proper idiom is "put through the wringer" (since a wringer is/was a device to squeeze water out of a wet mop or wet laundry. Squeezing someone through a pair of rollers is particularly evocative.)

Although you do see "nerve-wracking" or "wracking" (i.e. wrecking) one's brain, the more traditional "racking" (literally to torture by stretching on a medieval rack) seems more appropriate (although the term "nervous wreck" is common.) Shakespeare may have exploited the pun of "wrack" vs. "rack," so perhaps we can also.

"Security breaches" and "security breeches" sound alike but have somewhat dissimilar meanings. Network and system administrators might consider donning the latter in preparation for the former.

Re: A spellchecker used to be a major feat of software engineering (2008)

#15
A good spell checker is still a hard engineering problem, despite the hardware progress.

Just a hash map ain't gonna work. The only reason spell checking is perceived as a solved problem is availability of libraries. Here's an open source example https://github.com/hunspell/hunspell way above 10k lines of code.

I speak 4 languages, and in my experience what's in Microsoft Office is the best one I used so far.

Re: A spellchecker used to be a major feat of software engineering (2008)

#16
Fast forward to today. A program to load /usr/share/dict/words into a hash table is 3-5 lines of Perl or Python, depending on how terse you mind being. [...] That's progress.

I was curious -- This Python script consumes 9.5 MB of RAM on my Mac, which is a whole lot of 1980s-era PCs. Sure, in most cases this one decision to use a terse but unoptimized data structure won't matter much on a modern computer, but it adds up!

Re: A spellchecker used to be a major feat of software engineering (2008)

#17
post #3

Google's spell checker still is a major feat of software engineering ;-) It must handle every language, handle new words constantly being coined, have low enough latency and high enough throughput to run it on every web search. (Well, caching probably solves 2/3 of traffic.)

It's also present in auto complete

Re: A spellchecker used to be a major feat of software engineering (2008)

#18

Arguably detecting typographical (or transcription) errors is still non-trivial today since a) edit distance is NP complete and b) selecting the correct spelling often depends on grammar as well as semantic context. For example, consider the erroneous phrase "he was put through the ringer." Although "ringer" matches a spelling in the dictionary, it doesn't make sense semantically (a "ringer" being a device that rings…

This isn't in any way a criticism, but I found it really amusing that your post contained almost exactly the error you were describing ("hat") before correction.

Re: A spellchecker used to be a major feat of software engineering (2008)

#19
post #3

Google's spell checker still is a major feat of software engineering ;-) It must handle every language, handle new words constantly being coined, have low enough latency and high enough throughput to run it on every web search. (Well, caching probably solves 2/3 of traffic.)

It's also present in auto complete

True - that's even more requests per second than search is!

Re: A spellchecker used to be a major feat of software engineering (2008)

#20
post #5

Here's an article that people might be interested in. It gives a bit more detail: https://web.archive.org/web/20100706052342/http://www.spelli... I'm particularly interested in this one, and I'm curious about how useful something like this would be to use. > The second does not use a dictionary at all (Morris & Cherry 1975). Like the previous method, it divides the text into trigrams, but it creates a table of these,…

A while ago, I was considering building a text editor that ran SQL for a very specific purpose at work. I mused about adding a syntax checker/suggested. I wonder if the above approach might be a better approach to a look up table.
Post reply on HN