Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

github.com

Post reply on HN
Print view View original

121–130 of 178 posts

Re: Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

#121

Post by aidenn0 »

fred123 wrote: post #28
aidenn0 wrote: post #22

Earlier quoted context omitted.

> I assume it is too slow in CPU-only mode. So you don't have to assume: I gave up after running on 8 cores (Ryzen 7 2700) for 10 days for a single page.

Something wrong with your setup. It should be less than 30 s per page with your hardware

Huh, I tried with the version from pip (instead of my package manager) and it completes in 22s. Output on the only page I tested is considerably worse than tesseract, particularly with punctuation. The paragraph detection seemed to not work at all, rendering the entire thing on a single line.

Even worse for my uses, Tesseract had two mistakes on this page (part of why I picked it), and neither of them were correctly read by EasyOCR.

Partial list of mistakes:

1. Missed several full-stops at the end of sentences

2. Rendered two full-stops as colons

3. Rendered two commas as semicolons

4. Misrendered every single em-dash in various ways (e.g. "\_~")

5. Missed 4 double-quotes

6. Missed 3 apostrophes, including rendering "I'll" as "Il"

7. All 5 exclamation points were rendered as a lowercase-ell ("l"). Tesseract got 4 correct and missed one.

Re: Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

#122

Post by vunderba »

I did something similar about a decade ago because I was using tesseract to OCR Chinese.

Part of the problem is that if you use Tesseract to recognize English text it's much easier to clean it up afterwards because if it makes a mistake it's usually in only a single character, and you can use Levenstein distance to spellcheck and fix which will help a lot with the accuracy.

Logographic languages such as Chinese present a particular challenge to "conventional post-processing" having many words represented as two characters and often a lot of words as a single "glyph". This is particularly difficult because if it gets that glyph wrong there's no way to obvious way to detect the identification error.

The solution was to use image magick to "munge" the image (scale, normalize, threshold, etc), send each of these variations to tesseract, and then use a Chinese-corpus based Markov model to score the statistical frequency of the recognized sentence and vote on a winner.

It made a significant improvement in accuracy.

Re: Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

#123

Post by sumedh »

kbyatnal wrote: post #71

"real improvements came from adjusting the prompts to make things clearer for the model, and not asking the model to do too much in a single pass" This is spot on, and it's the same as how humans behave. If you give a human too many instructions at once, they won't follow all of them accurately. I spend a lot of time thinking about LLMs + documents, and in my opinion, as the models get better, OCR is soon going to be…

Is there a pricing page?

Re: Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

#124

Post by sumedh »

ajcp wrote: post #38
__jl__ wrote: post #21

I think Gemini Flash 1.5 is the best closed-source model for this. Very cheap. Particularly compared to GPT4o-mini, which is priced the same as GPT4 for image input tokens. Performance and speed is excellent. I convert each pdf page to an image and send one request per page to Flash (asynchronously). The prompt asks for markdown output with specific formatting guidelines. For my application (mainly pdf slideshows wit…

> I convert each pdf page to an image and send one request per page to Flash Why convert? Flash 1.5 accepts whole PDFs just fine. It will also increase the models response accuracy. Context: I have found Flash 1.5 is excellent and stable for this kind of use-case. Even at a non-EA price-point it's incredibly cheap, especially when utilizing Batch Prediction Jobs (50% discount!).

> Flash 1.5 accepts whole PDFs just fine.

Sometimes models cannot extract the text from the pdf in that case you need to use give the image of the page.

Re: Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

#125

Post by abdullahkhalids »

vunderba wrote: post #122

I did something similar about a decade ago because I was using tesseract to OCR Chinese. Part of the problem is that if you use Tesseract to recognize English text it's much easier to clean it up afterwards because if it makes a mistake it's usually in only a single character, and you can use Levenstein distance to spellcheck and fix which will help a lot with the accuracy. Logographic languages such as Chinese prese…

People's handwriting vary widely, and a human reading someone's writing faces the same problems you mention. For a language like English, humans also decipher unrecognized characters by looking at what letter would fix the word or what word would fit in the sentence, etc.

Surely handwriting quality distribution for Chinese is not too far off from the rest of the world. How do Chinese humans read handwritten text written by someone with a bad handwriting?

Re: Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

#127

Post by [deleted] »

vunderba wrote: post #122

I did something similar about a decade ago because I was using tesseract to OCR Chinese. Part of the problem is that if you use Tesseract to recognize English text it's much easier to clean it up afterwards because if it makes a mistake it's usually in only a single character, and you can use Levenstein distance to spellcheck and fix which will help a lot with the accuracy. Logographic languages such as Chinese prese…

[deleted]

Re: Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

#128

Post by cowsaymoo »

__jl__ wrote: post #21

I think Gemini Flash 1.5 is the best closed-source model for this. Very cheap. Particularly compared to GPT4o-mini, which is priced the same as GPT4 for image input tokens. Performance and speed is excellent. I convert each pdf page to an image and send one request per page to Flash (asynchronously). The prompt asks for markdown output with specific formatting guidelines. For my application (mainly pdf slideshows wit…

Cheap for now. One day, once the market shares balance out, the cloud spend will increase. Local LLMs may be important to prioritize for code that may be running after multiple subscription cycles into the future.

Edit: oh you best wrote closed-source model whoops

Re: Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

#129

Post by eigenvalue »

reissbaker wrote: post #126

Even simpler, you can convert each PDF page to a PNG and ask gpt4 to simply transcribe the image. In my experience it's extremely accurate, more so than Tesseract or classic OCR.

That would cost like 100x as much though.

Re: Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs

#130

Post by geraldog »

geraldog wrote: post #110

This is a wonderful idea, but while I appreciate the venerable Tesseract I also think it's time to move on. I personally use PaddlePaddle and have way better results to correct with LLMs. With PPOCRv3 I wrote a custom Python implementation to cut books at word-level by playing with whitespace thresholds. It works great for the kind of typesetting found generally on books, with predictable whitespace threshold between…

320 x 48 pixels actually.
Post reply on HN