Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-OCR – A multimodal OCR model for complex document understanding

github.com

Post reply on HN
Print view View original

51–60 of 82 posts

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#51

Post by chaps »

kergonath wrote: post #46
chaps wrote: post #33

Earlier quoted context omitted.

For my workflows, layout extraction has been so inconsistent that I've stopped attempting to use it. It's simpler to just throw everything into postgis and run intersection checks on size-normalized pages.

Interesting. What kind of layout do you have? My documents have one or two-column layouts, often inconsistently across pages or even within a page (which tripped older layout detection methods). Most models seem to understand that well enough so they are good enough for my use case.

Documents that come from FOIA. So, some scanned, some not. Lots of forms and lots of hand writing to add info that the form format doesn't recognize. Lots of repeated documents, but lots of one-off documents that have high signal.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#52

Post by mixedmath »

coder543 wrote: post #3

There are a bunch of new OCR models. I’ve also heard very good things about these two in particular: - LightOnOCR-2-1B: https://huggingface.co/lightonai/LightOnOCR-2-1B - PaddleOCR-VL-1.5: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5 The OCR leaderboards I’ve seen leave a lot to be desired. With the rapid release of so many of these models, I wish there were a better way to know which ones are actually the be…

Are there leaderboards that you follow or trust?

Also, do you have preferred OCR models in your experience? I've had some success with dots.OCR, but I'm only beginning to need to work with OCR.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#53

Post by renewiltord »

aliljet wrote: post #2

This is actually the thing I really desperately need. I'm routinely analyzing contracts that were faxed to me, scanned with monstrously poor resolution, wet signed, all kinds of shit. The big LLM providers choke on this raw input and I burn up the entire context window for 30 pages of text. Understandable evals of the quality of these OCR systems (which are moving wicked fast) would be helpful... And here's the kicke…

I’m sure you’ve tried all this but you’ve tried inter-rater agreement via multiple attempts on same LLM vs different LLM? Perhaps your system would work better if you ran it through 5 models 3 times and then highlighted diffs for human chooser.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#54

Post by philipkglass »

coder543 wrote: post #3

There are a bunch of new OCR models. I’ve also heard very good things about these two in particular: - LightOnOCR-2-1B: https://huggingface.co/lightonai/LightOnOCR-2-1B - PaddleOCR-VL-1.5: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5 The OCR leaderboards I’ve seen leave a lot to be desired. With the rapid release of so many of these models, I wish there were a better way to know which ones are actually the be…

I have been trying to catch up with recent OCR developments too. My documents have enough special requirements that public benchmarks didn't tell me enough to decide. Instead I'm building a small document OCR project with visualization tools for comparing bounding boxes, extracted text, region classification, etc. GLM-OCR is my favorite so far [1]. Apple's VisionKit is very good at text recognition, and fast, but it doesn't do high level layout detection and it only works on Apple hardware. It's another useful source of data for cross-validation if you can run it.

This project has been pretty easy to build with agentic coding. It's a Frankenstein monster of glue code and handling my particular domain requirements, so it's not suitable for public release. I'd encourage some rapid prototyping after you've spent an afternoon catching up on what's new. I did a lot of document OCR and post-processing with commercial tools and custom code 15 years ago. The advent of small local VLMs has made it practical to achieve higher accuracy and more domain customization than I would have previously believed.

[1] If you're building an advanced document processing workflow, be sure to read the post-processing code in the GLM code repo. They're doing some non-trivial logic to fuse layout areas and transform text for smooth reading. You probably want to store the raw model results and customize your own post-processing for uncommon languages or uncommon domain vocabulary. Layout is also easier to validate if you bypass their post-processing; it can make some combined areas "disappear" from the layout data.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#55

Post by coder543 »

mixedmath wrote: post #52
coder543 wrote: post #3

There are a bunch of new OCR models. I’ve also heard very good things about these two in particular: - LightOnOCR-2-1B: https://huggingface.co/lightonai/LightOnOCR-2-1B - PaddleOCR-VL-1.5: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5 The OCR leaderboards I’ve seen leave a lot to be desired. With the rapid release of so many of these models, I wish there were a better way to know which ones are actually the be…

Are there leaderboards that you follow or trust? Also, do you have preferred OCR models in your experience? I've had some success with dots.OCR, but I'm only beginning to need to work with OCR.

> Are there leaderboards that you follow or trust?

Not for OCR.

Regardless of how much some people complain about them, I really do appreciate the effort Artificial Analysis puts into consistently running standardized benchmarks for LLMs, rather than just aggregating unverified claims from the AI labs.

I don't think LMArena is that amazing at this point in time, but at least they provide error bars on the ELO and give models the same rank number when they're overlapping.

> Also, do you have preferred OCR models in your experience?

It's a subject I'm interested in, but I don't have enough experience to really put out strong opinions on specific models.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#56

Post by mwcampbell »

Stagnant wrote: post #36
alaanor wrote: post #16

There was so many OCR models released in the past few months, all VLM models and yet none of them handle Korean well. Every time I try with a random screenshot (not a A4 document) they just fail at a "simple" task. And funnily enough Qwen3 8B VL is the best model that usually get it right (although I couldn't get the bbox quite well). Even more funny, whatever is running on an iphone locally on cpu is insanely good,…

Chrome ships a local OCR model for text extraction from PDFs which is better than any of the VLM or open source OCR models i've tried. I had a few hundred gigs of old newspaper scans and after trying all the other options I ended up building a wrapper around the DLL it uses to get the text and bboxes. Performance and accuracy on another level compared to tesseract, and while VLM models sometimes produced good results…

What's the name of this DLL? I assume it's separate from the monster chrome.dll, and that the model is proprietary.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#57

Post by dweekly »

coder543 wrote: post #3

There are a bunch of new OCR models. I’ve also heard very good things about these two in particular: - LightOnOCR-2-1B: https://huggingface.co/lightonai/LightOnOCR-2-1B - PaddleOCR-VL-1.5: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.5 The OCR leaderboards I’ve seen leave a lot to be desired. With the rapid release of so many of these models, I wish there were a better way to know which ones are actually the be…

I'm going to be the obnoxious person who asks you to please create this leaderboard because you care and have a modicum of knowledge in this space.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#58

Post by staticman2 »

coder543 wrote: post #40
staticman2 wrote: post #38

Earlier quoted context omitted.

Gemini Pro 3 seems to be built for handling multiple page PDFs. I can feed it a multiple page PDF and tell it to convert it to markdown and it does this well. I don't need to load the pages one at a time as long as I use the PDF format. (This was tested on A.i. studio but I think the API works the same way).

It's not that they can't do multiple pages... but did you compare against doing one page at a time? How many pages did you try in a single request? 5? 50? 500? I fully believe that 5 pages of input works just fine, but this does not scale up to larger documents, and the goal of OCR is usually to know what is actually written on the page... not what "should" have been written on the page. I think a larger number of pa…

I've been doing small PDFs- usually 5 or 6 pages in length.

I never tested Gemini 3 PDF OCR compared to individual images but I can say it processes a small 6 page PDF better than the retired Gemini 1.5 or 2 did individual images.

I agree that OCR and analysis should be two separate steps.

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#59

Post by fudged71 »

kergonath wrote: post #26
StableAlkyne wrote: post #15

Earlier quoted context omitted.

How do these compare to something like Tesseract? I remember that one clearing the scoreboard for many years, and usually it's the one I grab for OCR needs due to its reputation.

Tesseract does not understand layout. It’s fine for character recognition, but if I still have to pipe the output to a LLM to make sense of the layout and fix common transcription errors, I might as well use a single model. It’s also easier for a visual LLM to extract figures and tables in one pass.

I don't know how, but PyMuPDF4LLM is based on Tessaract and has GNN-based layout detection

Re: GLM-OCR – A multimodal OCR model for complex document understanding

#60

Post by Stagnant »

mwcampbell wrote: post #56
Stagnant wrote: post #36

Earlier quoted context omitted.

Chrome ships a local OCR model for text extraction from PDFs which is better than any of the VLM or open source OCR models i've tried. I had a few hundred gigs of old newspaper scans and after trying all the other options I ended up building a wrapper around the DLL it uses to get the text and bboxes. Performance and accuracy on another level compared to tesseract, and while VLM models sometimes produced good results…

What's the name of this DLL? I assume it's separate from the monster chrome.dll, and that the model is proprietary.

chrome_screen_ai.dll is the name of the dll (libchromescreenai.so on linux) and yes it is proprietary. It isn't included by default, Chrome uses its component service to download it automatically when you open a PDF file that doesn't have pre-existing OCR'd text on it. You can download it separately from here: https://chrome-infra-packages.appspot.com/p/chromium/third_p...
Post reply on HN