Live data from Hacker News

How to Write a Spelling Corrector

norvig.com

81–90 of 133 posts

Re: How to Write a Spelling Corrector

#82
post #70
post #53

Earlier quoted context omitted.

Classified reports. This technique is called the Canary Trap, with the name (but not the technique) coined by Tom Clancy. https://en.wikipedia.org/wiki/Canary_trap

Okay, If you broaden it enough I'm sure you can find some use for almost everything. I was thinking of more of a mainstream application.

[deleted]

Re: How to Write a Spelling Corrector

#83
post #67

I couldn't help but read this and think about all the "coding" initiatives I've seen in K-12 and shake my head. What Norvig is doing is what we should be teaching. He is tackling this seemingly REALLY hard problem by thinking about it methodically, translating some intuition into code, carefully constructing an argument about how to solve it, and ways that it could be extended. This is what actual engineers look like…

But what you should realize is that the person who wrote this beautiful piece of code is someone who is at the peak of their career, a thinker, an artist, and that it must be ok to not ever rise to his level, because this is close to a master-piece. Few of these geniuses become professors, is just a guess, because no matter how quirky you are you always get a job somewhere.

I think Norvig's actually improved substantially since he wrote this — he wasn't at the peak of his career! But yes, it's a masterpiece, and yes, it's okay to not ever rise to that level, because even if nothing you ever create is as worthwhile as this, it's still worthwhile. Hacking is fun! And sometimes very useful, too.

(Also note that, despite appearances, this isn't the work of one man. Norvig didn't design Python and didn't invent the tabling technique for speeding up search; and Darius found a couple of bugs in the code. That doesn't mean it's not an authentic Norvig masterpiece.)

Re: How to Write a Spelling Corrector

#86
post #83
post #67

Earlier quoted context omitted.

But what you should realize is that the person who wrote this beautiful piece of code is someone who is at the peak of their career, a thinker, an artist, and that it must be ok to not ever rise to his level, because this is close to a master-piece. Few of these geniuses become professors, is just a guess, because no matter how quirky you are you always get a job somewhere.

I think Norvig's actually improved substantially since he wrote this — he wasn't at the peak of his career! But yes, it's a masterpiece, and yes, it's okay to not ever rise to that level, because even if nothing you ever create is as worthwhile as this, it's still worthwhile. Hacking is fun! And sometimes very useful, too. (Also note that, despite appearances, this isn't the work of one man. Norvig didn't design Pyth…

Sir, btw, I need a rule that I can use intuitively when deciding on wether to concatenate two words and when to put a dash between them. Do you have one?

Re: How to Write a Spelling Corrector

#87
post #3

Spell chequer Martha Snow Eye halve a spelling chequer It came with my pea sea It plainly marques four my revue Miss steaks eye kin knot sea. Eye strike a quay and type a word And weight four it two say Weather eye am wrong oar write It shows me strait a weigh. As soon as a mist ache is maid It nose bee fore two long And eye can put the error rite It's rare lea ever wrong. Eye have run this poem threw it I am shore y…

I had a surprisingly hard time reading this; but I don't subvocalize, and words are 'things' to me (for example, your/you're is read differently in my head and makes me very sensitive to that mistake).

Re: How to Write a Spelling Corrector

#89
post #18
post #3

Spell chequer Martha Snow Eye halve a spelling chequer It came with my pea sea It plainly marques four my revue Miss steaks eye kin knot sea. Eye strike a quay and type a word And weight four it two say Weather eye am wrong oar write It shows me strait a weigh. As soon as a mist ache is maid It nose bee fore two long And eye can put the error rite It's rare lea ever wrong. Eye have run this poem threw it I am shore y…

In a way, I'm surprised I can read this so easily. Do our brains read by converting text to sounds, and then parsing the sounds?

I think many people do have brains that operate this way, probably because their understanding of language is grounded in conversation as a first experience. I suspect this is the reason you see confusion with "your, you're" and "there, their, they're."

I don't parse language in quite that manner, and even in speaking I have a different "mouth feel" for those homonyms. This creates a modest inversion of the idea of spelling errors for me, in that people who truly do say homonyms in such a way that to my hearing it is exactly the same require me to parse for context.

Re: How to Write a Spelling Corrector

#90
post #47
post #18

Earlier quoted context omitted.

In a way, I'm surprised I can read this so easily. Do our brains read by converting text to sounds, and then parsing the sounds?

Early reading is taught that way. You may have heard of "Phonics" as an element of literacy? Full literacy in alphabetic-phonetic languages includes the steps of (1) no longer sounding out words aloud and (2) no longer sounding out most words in your head, but rather grasping the shape of the word immediately.

It's the jump from word-as-sound to word-as-symbol, which I think a lot of people never make, partially because it's not particularly useful in their lives.

I suspect those involved in programming skew very much towards the word-as-symbol, but whether that is causative or correlative I can't say.

Post reply on HN