Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-OCR – A multimodal OCR model for complex document understanding

github.com

Post reply on HN
Print view View original

21–30 of 82 posts

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#21

Post by fzysingularity »

ghrl wrote: post #19
alaanor wrote: post #16

There was so many OCR models released in the past few months, all VLM models and yet none of them handle Korean well. Every time I try with a random screenshot (not a A4 document) they just fail at a "simple" task. And funnily enough Qwen3 8B VL is the best model that usually get it right (although I couldn't get the bbox quite well). Even more funny, whatever is running on an iphone locally on cpu is insanely good,…

I remember someone building a meme search engine for millions of images using a cluster of used iPhone SE's because of Apple's very good and fast OCR capabilities. Quite an interesting read as well: https://news.ycombinator.com/item?id=34315782

Apple OCR even on the Mac is insanely good, in fact way better than AWS textract/GCP cloud vision OCR.

Any idea what model is being used?

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#22

Post by [deleted] »

StableAlkyne wrote: post #15
coder543 wrote: post #3

There are a bunch of new OCR models. I’ve also heard very good things about these two in particular: - LightOnOCR-2-1B: https://huggingface.co/lightonai/LightOnOCR-2-1B - PaddleOCR-VL-1.5: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5 The OCR leaderboards I’ve seen leave a lot to be desired. With the rapid release of so many of these models, I wish there were a better way to know which ones are actually the be…

How do these compare to something like Tesseract? I remember that one clearing the scoreboard for many years, and usually it's the one I grab for OCR needs due to its reputation.

[deleted]

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#24

Post by bugglebeetle »

I tested this pretty extensively and it has a common failure mode that prevents me from using: extracting footnotes and similar from the full text of academic works. For some reason, many of these models are trained in a way that results in these being excluded, despite these document sections often containing import details and context. Both versions of DeepseekOCR have the same problem. Of the others I’ve tested, dot-ocr in layout mode works best (but is slow) and then datalab’s chandra model (which is larger and has bad license constraints).

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#25

Post by droidjj »

bugglebeetle wrote: post #24

I tested this pretty extensively and it has a common failure mode that prevents me from using: extracting footnotes and similar from the full text of academic works. For some reason, many of these models are trained in a way that results in these being excluded, despite these document sections often containing import details and context. Both versions of DeepseekOCR have the same problem. Of the others I’ve tested, d…

I have been looking for an OCR model that can accurately handle footnotes. It’s essential for processing legal texts in particular, which often have footnotes that break across pages. Sadly I’ve yet to encounter a good solution.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#26

Post by kergonath »

StableAlkyne wrote: post #15
coder543 wrote: post #3

There are a bunch of new OCR models. I’ve also heard very good things about these two in particular: - LightOnOCR-2-1B: https://huggingface.co/lightonai/LightOnOCR-2-1B - PaddleOCR-VL-1.5: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5 The OCR leaderboards I’ve seen leave a lot to be desired. With the rapid release of so many of these models, I wish there were a better way to know which ones are actually the be…

How do these compare to something like Tesseract? I remember that one clearing the scoreboard for many years, and usually it's the one I grab for OCR needs due to its reputation.

Tesseract does not understand layout. It’s fine for character recognition, but if I still have to pipe the output to a LLM to make sense of the layout and fix common transcription errors, I might as well use a single model. It’s also easier for a visual LLM to extract figures and tables in one pass.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#27

Post by ks2048 »

I've been trying different OCR models on what should be very simple - subtitles (these are simple machine-rendered text). While all models do very well (95+% accuracy), I haven't seen a model not occasionally make very obvious mistakes. Maybe it will take a different approach to get the last 1%...

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#28

Post by chaps »

StableAlkyne wrote: post #15
coder543 wrote: post #3

There are a bunch of new OCR models. I’ve also heard very good things about these two in particular: - LightOnOCR-2-1B: https://huggingface.co/lightonai/LightOnOCR-2-1B - PaddleOCR-VL-1.5: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5 The OCR leaderboards I’ve seen leave a lot to be desired. With the rapid release of so many of these models, I wish there were a better way to know which ones are actually the be…

How do these compare to something like Tesseract? I remember that one clearing the scoreboard for many years, and usually it's the one I grab for OCR needs due to its reputation.

Tesseract v4 when it was released was exceptionally good and blew everything out of the water. Have used it to OCR millions of pages. Tbh, I miss the simplicity of tesseract.

The new models are similarly better compared to tesseract v4. But what I'll say is that don't expect new models to be a panacea for your OCR problems. The edge case problems that you might be trying to solve (like, identifying anchor points, or identifying shared field names across documents) are still pretty much all problematic still. So you should still expect things like random spaces or unexpected characters to jam up your jams.

Also some newer models tend to hallucinate incredibly aggressively. If you've ever seen an LLM get stuck in an infinite, think of that.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#30

Post by kergonath »

HPsquared wrote: post #12
coder543 wrote: post #6

Earlier quoted context omitted.

If you want OCR with the big LLM providers, you should probably be passing one page per request. Having the model focus on OCR for only a single page at a time seemed to help a lot in my anecdotal testing a few months ago. You can even pass all the pages in parallel in separate requests, and get the better quality response much faster too. But, as others said, if you can't afford mistakes, then you're going to need a…

You could maybe then do a second pass on the whole text (as plain text not OCR) to look for likely mistakes.

This is not always easy. The models I tried were too helpful and rewrote too much instead of fixing simple typos. When I tried I ended up with huge prompts and I still found sentences where the LLM was too enthusiastic. I ended up applying regexes with common typos and accepted some residual errors. It might be better now, though. But since then I’ve moved to all-in-one solutions like Mathpix and Mistral-OCR which are quite good for my purpose.
Post reply on HN