Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Write a Spelling Corrector

norvig.com

Post reply on HN
Print view View original

121–130 of 133 posts

Re: How to Write a Spelling Corrector

#121

Post by emmelaich »

ChicagoBoy11 wrote: post #52

I couldn't help but read this and think about all the "coding" initiatives I've seen in K-12 and shake my head. What Norvig is doing is what we should be teaching. He is tackling this seemingly REALLY hard problem by thinking about it methodically, translating some intuition into code, carefully constructing an argument about how to solve it, and ways that it could be extended. This is what actual engineers look like…

Yes, my son's High School Software Course has a lot of old dusty design content (data flow diagrams), chunks of soft content - "social impact of computers".

And the actual coding is a Visual Basic forms and a bit of html/javascript.

Very little in the way of what I would call proper coding: data structures, algorithms.

Re: How to Write a Spelling Corrector

#122

Post by s-phi-nl »

clentaminator wrote: post #24
lb1lf wrote: post #3

Spell chequer Martha Snow Eye halve a spelling chequer It came with my pea sea It plainly marques four my revue Miss steaks eye kin knot sea. Eye strike a quay and type a word And weight four it two say Weather eye am wrong oar write It shows me strait a weigh. As soon as a mist ache is maid It nose bee fore two long And eye can put the error rite It's rare lea ever wrong. Eye have run this poem threw it I am shore y…

Love it. It also reminds me of the poem The Chaos, by Gerard Nolst Trenité: http://ncf.idallen.com/english.html

Along similar lines, "The King's English" http://holyjoe.org/poetry/anonA.htm

Re: How to Write a Spelling Corrector

#124

Post by taneq »

xapata wrote: post #68
drauh wrote: post #65

Earlier quoted context omitted.

By that reasoning, we should not teach kids how to spell, or about punctuation, and just aim for them writing essays/stories/novels.

That's how I wish foreign language were taught. I don't care about punctuation. I just want to communicate without embarrassing myself.

My sister once told me that learning a (human) language seems boring until you want to know what that cute boy is saying.

Re: How to Write a Spelling Corrector

#125

Post by xapata »

taneq wrote: post #124
xapata wrote: post #68

Earlier quoted context omitted.

That's how I wish foreign language were taught. I don't care about punctuation. I just want to communicate without embarrassing myself.

My sister once told me that learning a (human) language seems boring until you want to know what that cute boy is saying.

That's basically how I learned French.

Re: How to Write a Spelling Corrector

#127

Post by ThePawnBreak »

bsenftner wrote: post #12

Although not a spell checker specifically, I wrote an offensive language filter for a chat system used by the National Hockey League for a period when they hosted communal chat rooms during televised games. The architecture I used is completely different from what is described here, but the goals are very similar. I had to handle any curse word in any language, including curses from one language translated into anoth…

> but I remember using Perfect Hash to generate a compiled hash table for the dictionary, and then a trie to organize the dictionary lookups

But... but programmers on HN told me that tries are just useless pieces of trivia used to fail people in interviews, and if you need them a library is just going to do it for you in the most efficient possible way.

Re: How to Write a Spelling Corrector

#128

Post by clentaminator »

s-phi-nl wrote: post #122
clentaminator wrote: post #24

Earlier quoted context omitted.

Love it. It also reminds me of the poem The Chaos, by Gerard Nolst Trenité: http://ncf.idallen.com/english.html

Along similar lines, "The King's English" http://holyjoe.org/poetry/anonA.htm

Thanks for posting this. It's just as much of a tongue twister as The Chaos :)

Re: How to Write a Spelling Corrector

#129

Post by mrweasel »

jomamaxx wrote: post #114

I've done this before. Sadly - the problem is way harder. First - you get much better results by using language models, n-grams etc. to predict the likely hood of words given previous words. That can be hard. The really hard part comes down to language that people use. Colloquialisms, proper names, and mixed-languages ... make this stuff really, really hard. Getting it 'mostly right' is not hard. Getting it really go…

Writing spell checkers is also hard in language that e.g. doesn't have spaces between composite nouns. In essence it makes it possible to make up a perfectly valid new word. We expect the spell checker to be able to recognise that word, because it knows the two words that the composite is made of.

Finnish is probably even worse because of it's conjugation.

Re: How to Write a Spelling Corrector

#130

Post by kybernetikos »

shanusmagnus wrote: post #100
drauh wrote: post #65

Earlier quoted context omitted.

By that reasoning, we should not teach kids how to spell, or about punctuation, and just aim for them writing essays/stories/novels.

"If you want to build a ship, don't drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea."

That's a lovely idea although I can't help noticing that I know of no shipbuilding companies based on the teaching of yearning while I do know of some that seem based on organising a workforce. I suspect that this is one of those ideas that while beautiful is far from being literally true.

How about "if you want your citizens to fund a space program, don't gather the workforce to build a spaceship but teach them to yearn for the vastness and potential of the void."

Post reply on HN