Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama-OCR: Document to Markdown

llamaocr.com

Post reply on HN
Print view View original

71–80 of 104 posts

Re: Llama-OCR: Document to Markdown

#71

Post by constantinum »

The problem with using LLMs for OCR is hallucinations. It makes it impossible to use in business use cases such as insurance, banking and health/medical — which demands high accuracy or predictable inaccuracy rate. Not to mention handling scale — processing millions of documents with speed and affordable costs.

For all the test use cases mentioned in this thread, I’d suggest trying LLMwhisperer. A general purpose text Pre-processor/OCR built for LLM consumption. https://pg.llmwhisperer.unstract.com

Re: Llama-OCR: Document to Markdown

#72

Post by cess11 »

alecco wrote: post #57

Is it possible to do this locally with open source software? I have a lot of accounting PDFs to convert but due to privacy concerns it should not run in the cloud.

Either you need to be somewhat tolerant when it comes to misinterpretations and hallucinations, or you'll be proofreading a lot.

A cheap hack is to push the documents through pdftotext from Poppler and if nothing or very little comes out, push them through OCRMyPDF and pipe it to pdftotext. If it's scanned you probably want some flags for deskewing and so on.

To make a bulk load of PDF mostly greppable it's a decent technique, to get every 0 as a 0 you're probably going to proofread every conversion.

Re: Llama-OCR: Document to Markdown

#73

Post by cess11 »

AmazingTurtle wrote: post #44

One can combine apache tika OCR and feed it together with the image into LLM to fix typos.

While I'm a fan of Tika a lot of people get queasy from Java and XML, they might be better served by their preferred scripting language and https://github.com/ocrmypdf/OCRmyPDF, which has the same OCR engine.

Re: Llama-OCR: Document to Markdown

#74

Post by [deleted] »

notsylver wrote: post #17

I've been doing a lot of OCR recently, mostly digitising text from family photos. Normal OCR models are terrible at it, LLMs do far better. Gemini Flash came out on top from the models I tested and it wasn't even close. It still had enough failures and hallucinations to make it faster to write it in by hand. Annoying considering how close it feels to working. This seems worse. Sometimes it replies with just the text,…

[deleted]

Re: Llama-OCR: Document to Markdown

#75

Post by [deleted] »

hrpnk wrote: post #67

Reading the Llama community license agreement, section "Redistribution and Use" I expected to find 'Built with Llama'. Is this not required? https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instr... links to the community license.

[deleted]

Re: Llama-OCR: Document to Markdown

#79

Post by Eisenstein »

alecco wrote: post #57

Is it possible to do this locally with open source software? I have a lot of accounting PDFs to convert but due to privacy concerns it should not run in the cloud.

I don't recommend using it for anything important unless you very diligently proofread it, but I made one that runs locally that I linked to elsewhere in this post:

* https://news.ycombinator.com/item?id=42155548

Re: Llama-OCR: Document to Markdown

#80

Post by danvk »

notsylver wrote: post #17

I've been doing a lot of OCR recently, mostly digitising text from family photos. Normal OCR models are terrible at it, LLMs do far better. Gemini Flash came out on top from the models I tested and it wasn't even close. It still had enough failures and hallucinations to make it faster to write it in by hand. Annoying considering how close it feels to working. This seems worse. Sometimes it replies with just the text,…

I've had really good luck recently running OCR over a corpus of images using gpt-4o. The most important thing I realized was that non-fancy data prep is still important, even with fancy LLMs. Cropping my images to just the text (excluding any borders) and increasing the contrast of the image helped enormously. (I wrote about this in 2015 and this post still holds up well with GPT: https://www.danvk.org/2015/01/07/finding-blocks-of-text-in-a...).

I also found that giving GPT at most a few paragraphs at a time worked better than giving it whole pages. Shorter text = less chance to hallucinate.

Post reply on HN