Ask HN: Open source OCR library?

It seems this is just built on top of Tesseract as well.

It's because the kind of massive training sets that are required for accurate speach/text recognition are difficult and time-consuming to amass. It's just too difficult for hobbiests at the moment.

It's kind of the same reason there is no good open source solid modelling software (like Solidworks or NX). The problem has been "solved" for years, but actually doing it is too mammoth a task for open-source software.

[OT] What about FreeCAD? It looks like good open source solid modelling software: http://freecadweb.org/

My solution with regards to bad facial detection in OpenCV is to do the following:

1. Use an LBP cascade on the picture. This is lower quality, higher false positives. Uses integer math so this is fast. Its named lbpcascade_frontalface.xml

2. Capture the regions of interest that the LBP cascade identifies with a face and throw into a vector. This means you can capture (potentially) arbitrary amount of faces. Of course, with OpenCV you are limited to a minimum of 28px by 28px minimum face.

3. Run the haar cascade for eye detection on the ROI's you saved in the vector. Ones that return eyes show a good match. Haar cascades are slower(because they use floats), but the reduction in pixels means its relatively fast. Its named haarcascade_eye_tree_eyeglasses.xml

I can maintain 20fps with this setup at 800x600 on a slow computer.

Yep. Back when I was at NYPL Labs[0], we developed a bit of an obsession with OCR. It turns out that so much of it depends on the context of the documents, original scan quality (unpaper and deskewing be damned), typography, how much handwriting you're willing to suffer (can someone please pick up the work of applying Convnets to large handwriting corpora that aren't just MNIST?).

A lot of this comes down to how much time you're willing to put into it to get it up and running and if you're willing to put in any effort to additionally train/refine a model.

(ps all the elements I'm about to mention here have been mentioned throughout this thread. I'm just providing a bit more context and bringing what I know together).

_________________

Gotta have something now: Tesseract

Not saying it's "bad" but it ain't as great as you want. Bindings for every language, lots of configs, documented well enough that it's 80% of the suggestions here. For what it is, it's a damn miracle. Most languages and alphabets and lots of kinds of fonts likely start giving you results somewhere between 80% and the Uncanny Valley immediately. You also get a pretty good Layout Analysis engine [1] for if you're working with complete pages of text. The existing models it ships with are robust, but if you want to get better outputs, retraining it is a real pain (you have to segment on a character-by-character basis). You're better off trying to apply a general preprocessing to clean the image or...

__________________

Gotta have a proof of concept this afternoon: Send just text to Tesseract

By now you've realized that asking an OCR engine to OCR a tree (or a picture of a tree alongside some text) somehow always comes back looking like a cat just napped on your keyboard. As alluded to in a few other places here, Just Give It Text! Tesseract (and most traditional OCR) was developed in a text document centric world where you could safely assume it's just a page of words, and sometimes you might have to deal with columns (that's where the Layout Analysis comes in). Probably not your real world.

Depending on your type of documents you might be able to develop a few hueristics for identifying text regions in the picture, then sending only those sections over to Tesseract. This'll dramaticlly help Tesseract out, but it can increase some of the complexity you'll have to juggle. You might be able to come up with some hueristics specific to your documents (which can be very good, especially if it lets you infer more information about those regions that you might want later).

You can also use something like Stroke-Width Transform (all the links I would use were graciously linked in this earlier comment [2]) which was discovered by Microsoft trying to spot text in the wild for their Street View efforts. ccv has a very nice SWT implementation [3], and their http server for the whole library [4] that with a bit of makefile finagleing can have a very nice SWT preprocessor -> Tesseract api over HTTP up in an hour or so.

Also it looks like the OpenOCR[5] project is now using SWT -> Tesseract with a nice HTTP API written in Go, conveniently packaged up for Docker and very well documented[6].

_____________________

Take me down the rabbit hole, but maybe get results in 2 days:

The roots of tesseract were planted by HP in 1985, so there had to be a better way at some point. OCRopus[7] was supposed to be the great state-of-the-art hope that would save us all. The approach was incredible and the results being published were great. But the documentation came in the form of a mind map[8]. Recently the project was picked up again by the original developer and rechristened OCRopy[9] and has garnered a pretty active and growing community in the past few months.

You'll have to do a bit more work here than just tesseract, but the LSTM neural network[10] approach completely blows away Tesseract's results with just a little training.

To get started, Dan Vanderkam's tutorial is excellent to start working with the out-of-the-box model[11] immediately.

But results get INCREDIBLE when you take some time to train your own model[12]. I provided Dan with the source images for the project[13], from The New York Public Library's historical collections, and the source text was in a font style the default model had never encountered before. But in an hour or two of transcribing the training data (training data in OCRopus is awesome because you just feed in a line at a time; no need to align and segment each letter like in Tesseract), he was getting an under 1% error rate!

Layout analysis isn't OCRopy's strongest suit, so you might get even better results if you pre-segment with something like SWT, but again, not completely necessary.

____________

That's it. Way too much for a Newsy Combinator comment, but a pretty decent tour of the big stuff in open ocr world. There's some magic state of the art going on inside Google, Microsoft, and Abbyy, but hopefully you'll be able to teach your computer to appreciate whatever it is you want it to read through these now.

[0]: http://nypl.org/labs

[1]: https://en.wikipedia.org/wiki/Document_layout_analysis

[2]: https://news.ycombinator.com/item?id=9776034

[3]: http://libccv.org/doc/doc-swt/ and http://libccv.org/lib/ccv-swt/

[4]: http://libccv.org/doc/doc-http/ and http://libccv.org/lib/ccv-serve/

[5]: https://github.com/tleyden/open-ocr

[6]: https://github.com/tleyden/open-ocr/wiki/Stroke-Width-Transf...

[7]: https://en.wikipedia.org/wiki/OCRopus

[8]: https://www.mindmeister.com/192257150/ocropus-overview-publi...

[9]: https://github.com/tmbdev/ocropy | all day all week

[10]: https://en.wikipedia.org/wiki/Long_short_term_memory

[11]: http://www.danvk.org/2015/01/09/extracting-text-from-an-imag... | though you've got to download the model separately because it's too big for github

[12]: http://www.danvk.org/2015/01/11/training-an-ocropus-ocr-mode... [13]: https://www.oldnyc.org/