Live data from Hacker News

How to write a spelling corrector (2016)

norvig.com

41–50 of 87 posts

Re: How to write a spelling corrector (2016)

#41
post #15

What I would like to know is why the spelling correction suggestion of Apple iOS are so bad. Why can they get along with this ? I used to disable it. It can't even make a suggestion when there is only one missing character.

Personally, I get confused why the spell-checker for Chrome is so bad. I so often type words that it recognizes are wrong, but has no suggestions at all for, when the correct word is just a single character off.

elipsis (ellipsis), elipses (ellipses), independant (independent), disserning (discerning)...

How can it not find the correct word?

Re: How to write a spelling corrector (2016)

#42

The unit tests worry me: assert len(WORDS) == 32192 assert sum(WORDS.values()) == 1115504 assert WORDS.most_common(10) == [ ('the', 79808), ('of', 40024), ('and', 38311), ('to', 28765), ('in', 22020), ('a', 21124), ('that', 12512), ('he', 12401), ('was', 11410), ('it', 10681)] assert WORDS['the'] == 79808 Those aren't testing the file open, or Counter, or read, but instead are tightly-coupling the tests to the exact…

"They" are Peter Norvig. I mention this not to appeal to authority, so much as to remind you that you're not reading the code of a junior engineer.

Re: How to write a spelling corrector (2016)

#43
post #38
post #32

I know a spelling corrector is not the same thing as a spelling checker, but this is too good an opportunity to pass to promote Martha Snow's hilarious poem 'Spell Chequer': Eye halve a spelling chequer It came with my pea sea It plainly marques four my revue Miss steaks eye kin knot sea. Eye strike a quay and type a word And weight four it two say Weather eye am wrong oar write It shows me strait a weigh. As soon as…

Thanks for sharing this. I found it difficult to parse the initial couple of lines because I was constantly attempting to read by attaching meaning to the spellings: "Eye halve" is a bit frightening in that sense. But then I realised I can read the text as sounds and almost ignore the spellings. Listening to what the sounds made in my head allowed for a much faster pace of comprehension because I didn't have to keep…

One part I can't "translate" is this:

    It plainly marques four my revue

Re: How to write a spelling corrector (2016)

#44
post #32

I know a spelling corrector is not the same thing as a spelling checker, but this is too good an opportunity to pass to promote Martha Snow's hilarious poem 'Spell Chequer': Eye halve a spelling chequer It came with my pea sea It plainly marques four my revue Miss steaks eye kin knot sea. Eye strike a quay and type a word And weight four it two say Weather eye am wrong oar write It shows me strait a weigh. As soon as…

My chequer marks chequer as wrong.

Re: How to write a spelling corrector (2016)

#45
post #43
post #38

Earlier quoted context omitted.

Thanks for sharing this. I found it difficult to parse the initial couple of lines because I was constantly attempting to read by attaching meaning to the spellings: "Eye halve" is a bit frightening in that sense. But then I realised I can read the text as sounds and almost ignore the spellings. Listening to what the sounds made in my head allowed for a much faster pace of comprehension because I didn't have to keep…

One part I can't "translate" is this: It plainly marques four my revue

"It plainly marks for my review"

Re: How to write a spelling corrector (2016)

#46
post #44
post #32

I know a spelling corrector is not the same thing as a spelling checker, but this is too good an opportunity to pass to promote Martha Snow's hilarious poem 'Spell Chequer': Eye halve a spelling chequer It came with my pea sea It plainly marques four my revue Miss steaks eye kin knot sea. Eye strike a quay and type a word And weight four it two say Weather eye am wrong oar write It shows me strait a weigh. As soon as…

My chequer marks chequer as wrong.

Damn spellcheck reverting to "English (US)".

Re: How to write a spelling corrector (2016)

#47

The unit tests worry me: assert len(WORDS) == 32192 assert sum(WORDS.values()) == 1115504 assert WORDS.most_common(10) == [ ('the', 79808), ('of', 40024), ('and', 38311), ('to', 28765), ('in', 22020), ('a', 21124), ('that', 12512), ('he', 12401), ('was', 11410), ('it', 10681)] assert WORDS['the'] == 79808 Those aren't testing the file open, or Counter, or read, but instead are tightly-coupling the tests to the exact…

This code is not written for production, it's just written to make you understand how the basic of this technology works. So I would say the unit tests have the exact same purpose: make the reader understand what the functions are doing (and not a real unit testing). Since you speak of value, the value Peter Norvig is trying to provide is making readers understand the principles, he's not trying to provide some monet…

The value he is trying to provide is also showing people by example how to write readable, idiomatic, functional Python code.

From that perspective, I agree the tightly coupled unit tests are a bit grating. Though they would not look out of place as doctests.

Re: How to write a spelling corrector (2016)

#48

The unit tests worry me: assert len(WORDS) == 32192 assert sum(WORDS.values()) == 1115504 assert WORDS.most_common(10) == [ ('the', 79808), ('of', 40024), ('and', 38311), ('to', 28765), ('in', 22020), ('a', 21124), ('that', 12512), ('he', 12401), ('was', 11410), ('it', 10681)] assert WORDS['the'] == 79808 Those aren't testing the file open, or Counter, or read, but instead are tightly-coupling the tests to the exact…

Those asserts seem like a pre-Jupyter form of narrative.

Re: How to write a spelling corrector (2016)

#49
post #38
post #32

I know a spelling corrector is not the same thing as a spelling checker, but this is too good an opportunity to pass to promote Martha Snow's hilarious poem 'Spell Chequer': Eye halve a spelling chequer It came with my pea sea It plainly marques four my revue Miss steaks eye kin knot sea. Eye strike a quay and type a word And weight four it two say Weather eye am wrong oar write It shows me strait a weigh. As soon as…

Thanks for sharing this. I found it difficult to parse the initial couple of lines because I was constantly attempting to read by attaching meaning to the spellings: "Eye halve" is a bit frightening in that sense. But then I realised I can read the text as sounds and almost ignore the spellings. Listening to what the sounds made in my head allowed for a much faster pace of comprehension because I didn't have to keep…

It does! A lot, imho.

Re: How to write a spelling corrector (2016)

#50
> I thought Dean and Bill, being highly accomplished engineers and mathematicians, would have good intuitions about how this process works. But they didn't, and come to think of it, why should they know about something so far outside their specialty?

That kind of speaks volumes about modern academics.

Post reply on HN