Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RNN-Based Handwriting Recognition in Gboard

ai.googleblog.com

Post reply on HN
Print view View original

1–10 of 23 posts

Re: RNN-Based Handwriting Recognition in Gboard

#2

Post by appleflaxen »

this is so awesome!

but how is it that we have RNN solutions for handwriting when we don't even have a standard, canned RNN for OCR?

I know tesseract and related projects exist, but when I've tried them they have been fairly brittle with lower accuracy than I was expecting. Accuracy was especially problematic for letter combinations like "-ing" that would consistently be recognized as "-mg".

Is there a good ML OCR library I'm missing?

Re: RNN-Based Handwriting Recognition in Gboard

#4

Post by pfortuny »

appleflaxen wrote: post #2

this is so awesome! but how is it that we have RNN solutions for handwriting when we don't even have a standard, canned RNN for OCR? I know tesseract and related projects exist, but when I've tried them they have been fairly brittle with lower accuracy than I was expecting. Accuracy was especially problematic for letter combinations like "-ing" that would consistently be recognized as "-mg". Is there a good ML OCR li…

Just a side comment: take into account that (as per the paper) there is temporal input in Gboard (i.e. the timestamp of each stroke is important).

You do not have that for ing, so the software does not know that the dot is “independent”).

Re: RNN-Based Handwriting Recognition in Gboard

#5

Post by reubenmorais »

The "Making it Work, On-device" paragraph makes it seem like TensorFlow Lite will easily get your model running fast on-device, but in reality RNNs aren't currently supported by the TFLite Converter and the TFLiteLSTMCell example is super slow for training, so this is actually based on proprietary code not available to mere mortals using open source TensorFlow. If you were to actually try reproducing this work, you'd have to use several workarounds, dig deep into the TensorFlow source code, and possibly still end up with a suboptimal TFLite model.

Don't get me wrong, in terms of deployability and flexibility for production usage, TensorFlow/TFLite is really good, specially compared to other frameworks, but Google tends to oversell the abilities of open-source TensorFlow significantly in their marketing material, and you only find out when you go and try doing it yourself.

Re: RNN-Based Handwriting Recognition in Gboard

#6

Post by lawrenceyan »

reubenmorais wrote: post #5

The "Making it Work, On-device" paragraph makes it seem like TensorFlow Lite will easily get your model running fast on-device, but in reality RNNs aren't currently supported by the TFLite Converter and the TFLiteLSTMCell example is super slow for training, so this is actually based on proprietary code not available to mere mortals using open source TensorFlow. If you were to actually try reproducing this work, you'd…

For industry/real world work, TensorFlow is best in class. It is far superior to any other existing framework. I agree that there are always areas for improvement, but the way you worded your comment makes it almost sound like TF is pretty subpar compared to other offerings.

The reality is more, TensorFlow is really the only option you have if you don’t want to build everything from scratch again. Whether that’s a good or bad thing, well at least it’s because TensorFlow is actually a good product and not because Google is preventing others from building their own / pushing others down.

Re: RNN-Based Handwriting Recognition in Gboard

#7

Post by dajohnson89 »

I just switched from Android to iPhone, and Gboard on iPhone doesn't have the translation function. It also doesn't have multiple languages -- if I want to switch languages I have to exit out of Gboard and use the default iOS keyboard. Anyone know why these features for Gboard are missing on iOS?

Re: RNN-Based Handwriting Recognition in Gboard

#9

Post by colde »

dajohnson89 wrote: post #7

I just switched from Android to iPhone, and Gboard on iPhone doesn't have the translation function. It also doesn't have multiple languages -- if I want to switch languages I have to exit out of Gboard and use the default iOS keyboard. Anyone know why these features for Gboard are missing on iOS?

It does have multiple languages. Clicking the settings icon right between the button to switch to numbers and the emoji button switches languages. You can also hold it down to go to settings.
Post reply on HN