Real time numbers recognition (MNIST) on an iPhone with CoreML
1–10 of 20 posts
Re: Real time numbers recognition (MNIST) on an iPhone with CoreML
#2Neat walkthrough!
Last year I actually made an applied-CoreML app to solve sudoku puzzles where MNIST came in very handy.
I wrote about it here: https://blog.prototypr.io/behind-the-magic-how-we-built-the-...
Re: Real time numbers recognition (MNIST) on an iPhone with CoreML
#3As someone with not much experience in ML, how to handle when there is no number present or if a number is present?
Re: Real time numbers recognition (MNIST) on an iPhone with CoreML
#4As someone with not much experience in ML, how to handle when there is no number present or if a number is present?
The predictions variable has a confidence value for each digit. You can put a cutoff and say if none is above a certain confidence, assume there's no number at all.
Re: Real time numbers recognition (MNIST) on an iPhone with CoreML
#5As someone with not much experience in ML, how to handle when there is no number present or if a number is present?
You mean to get either 0-9 or 'no number'? Here are two approaches:
1) Integrated. Represent 'no number' as class number 11 in the original model. Retrain it with this additional class (needs additional training data).
2) Cascading. Train a dedicated model for 'number' versus 'no number' (binary classifier), and use that in front of the original model.
Note that the MNIST data comes already extracted from original image, centered in fixed-size images of 28x28 pixels. In a practical ML application these steps would also need to be done before classification can be performed.
Re: Real time numbers recognition (MNIST) on an iPhone with CoreML
#6As someone with not much experience in ML, how to handle when there is no number present or if a number is present?
You mean to get either 0-9 or 'no number'? Here are two approaches: 1) Integrated. Represent 'no number' as class number 11 in the original model. Retrain it with this additional class (needs additional training data). 2) Cascading. Train a dedicated model for 'number' versus 'no number' (binary classifier), and use that in front of the original model. Note that the MNIST data comes already extracted from original im…
In the work shown in the article, the segmentation and centering of digits looks to be done by the user holding the camera. Which can be workable for some applications!
Re: Real time numbers recognition (MNIST) on an iPhone with CoreML
#7As someone with not much experience in ML, how to handle when there is no number present or if a number is present?
The predictions variable has a confidence value for each digit. You can put a cutoff and say if none is above a certain confidence, assume there's no number at all.
This could work, but it is important to note that a lot of ML algorithms trained in a closed domain (no "other" class) will be pretty bad at knowing what they don't know. This is an open problem in ML.
Re: Real time numbers recognition (MNIST) on an iPhone with CoreML
#8"Apple ... provides a ... helper library called coremltools that we can use to ... convert scikit-learn models, Keras and XGBoost models to CoreML"
Awesome.
Re: Real time numbers recognition (MNIST) on an iPhone with CoreML
#9Neat walkthrough! Last year I actually made an applied-CoreML app to solve sudoku puzzles where MNIST came in very handy. I wrote about it here: https://blog.prototypr.io/behind-the-magic-how-we-built-the-...
>After I scanned a wide variety of puzzles from each book, my server had stored about 600,000 images
600,000?!? Even divided by 81 that's over 7000! How long did this take?
Re: Real time numbers recognition (MNIST) on an iPhone with CoreML
#10Neat walkthrough! Last year I actually made an applied-CoreML app to solve sudoku puzzles where MNIST came in very handy. I wrote about it here: https://blog.prototypr.io/behind-the-magic-how-we-built-the-...
>After I scanned a wide variety of puzzles from each book, my server had stored about 600,000 images 600,000?!? Even divided by 81 that's over 7000! How long did this take?
A couple of afternoons.
I just hacked into my app's flow to upload a "scan" of the isolated puzzle to my server instead of slicing it and sending the component images to CoreML.
Then I sat there and flipped through page after page of Sudoku puzzles and scanned them from a few different angles each, sliced them in bulk on the server, and voila: data!