Live data from Hacker News

How to Write a Spelling Corrector

norvig.com

131–133 of 133 posts

Re: How to Write a Spelling Corrector

#131

Although not a spell checker specifically, I wrote an offensive language filter for a chat system used by the National Hockey League for a period when they hosted communal chat rooms during televised games. The architecture I used is completely different from what is described here, but the goals are very similar. I had to handle any curse word in any language, including curses from one language translated into anoth…

> but I remember using Perfect Hash to generate a compiled hash table for the dictionary, and then a trie to organize the dictionary lookups But... but programmers on HN told me that tries are just useless pieces of trivia used to fail people in interviews, and if you need them a library is just going to do it for you in the most efficient possible way.

Tries are lovely little datastructures, don't let anyone tell you any different. It's not like you'll use them every day, but they're nice to have when needed!

Re: How to Write a Spelling Corrector

#132
post #77

Is this how modern spell checkers actually work? I assumed they would use a heuristic trying to match common misspellings to their frequent corrections. That or a combination of heuristic and Bayes.

For one, I'm fairly sure that modern spell checkers use n-gram language models rather than a 1-gram model to decrease perplexity. Disclaimer: I skimmed the article.
Post reply on HN