Back to the Future of Handwriting Recognition

The linked demo is by far the most impressive thing I've seen all week. I wish a certain Microsoft chart editor was as easy and unfinicky to use as this demo from 1966 (52 years ago), and that's still one of the better editors out there.

This a cool exploration of technology, and I don't want to take away from that.

> The program was efficient enough to run in real-time on a IBM System/360 computer, and robust enough to properly identify 90 percent of the symbols drawn by first-time users.

I just want to point out that 90% accuracy is, from a user's point of view, awful handwriting recognition performance. It means you will be correcting on average about 10 words per paragraph! Even 99% percent accuracy is not nearly good enough to give people a sense that the computer is good at handwriting recognition.

I also want to point out the difficulty and danger in interpreting strokes when doing handwriting recognition.

In the last demo box, try writing a capital Y without lifting the pen. You'll have to go "up and down" one or both upper branches. Because of this, the recognizer will call it a K, A, or N even though it is obviously a Y when you're done.

This demo is constrained to only using one stroke per letter, but systems that permit multiple strokes still get into trouble when the strokes don't match what they are expecting--for example if you draw an X using 4 individual strokes outward from a central point.

This also happens with words. In Microsoft's handwriting recognition in Office in the early 2000s, writing the letters of a word out of order completely borked the recognition. For example writing "xample" and then going back and adding an "e" at the beginning would not produce a recognized word of "example."

My point with all of this is that there is a reason you probably don't do all your computing with natural handwriting. It's a surprisingly difficult problem. Users do not expect it to matter how they form letters and words on the page. And they have very low tolerance for correcting computer mistakes.

This is kind of interesting. I had a through about how to approach the handwriting recognition problem a few years back and surprisingly I though of this curvature based approach also. I never implemented it (too lazy to try...) but its cool to see how well something like that might work.

I did something like this in Visual Basic and submitted it to PC PLUS in the UK, back in the early 90's.

It was (yay!) published as recognit.bas (VB) and I'd be really happy if someone still has a copy.

It recognized just numbers but the basis of operation was similar to the linked article.

The human convention of written language it to interpret the symbols after they have been completed, not during the act of writing them.

A computer that interprets the behavior of writing, rather than the final symbols, is going to violate user expectations at some point.

Why? Because people do not always write as linearly as you might expect, especially when writing fast. They might drop or mis-write letters or words, then go back and fix it. Or quickly jot down just enough letters to remind themselves of what they heard, then go back and fill the rest in. A routine that interprets actions in order is going to have a hard time with actions that the user completes out of order.