Llama-Scan: Convert PDFs to Text W Local LLMs • Hacker News Board

Re: Llama-Scan: Convert PDFs to Text W Local LLMs

#71

Post by agentcoops » Mon, Aug 18, 2025, 11:26 AM UTC

jchw wrote: post #55

HocusLocus wrote: post #11

By 1990 Omnipage 3 and its successors were 'good enough' and with their compact dictionaries and letter form recognition were miracles of their time at ~300MB installed. In 2025 LLMs can 'fake it' using Trilobites of memory and Petaflops. It's funny actually, like a supercomputer being emulated in real time on a really fast Jacquard loom. By 2027 even simple hand held calculator addition will be billed in kilowatt-ho…

A bit ago I tried throwing a couple of random simple Japanese comics (think 4koma but I don't think either of the ones I threw in were actually 4 panels) from Pixiv into Gemma 3b on AI studio. - It transcribed all of the text, including speech, labels on objects, onomatopoeias in actions, etc. I did notice a kana was missing a diacritic in a transcription, so the transcriptions were not perfect, but pretty close actu…

100%. My sense is that many in this thread have never gone through the misery of trying to use classical OCR for non-English documents or where you can't control scan quality. I did a test recently with 18th-century German documents, written in a well-known and standardized but archaic script. The accuracy of classical models specifically trained on this corpus was an order of magnitude lower than GPT5. I haven't experimented personally or professionally with smaller models, but your experience makes me hopeful that we might even get this accurate OCR on phones sooner rather than later...

Re: Llama-Scan: Convert PDFs to Text W Local LLMs

#72

Post by agentcoops » Mon, Aug 18, 2025, 11:30 AM UTC

GaggiX wrote: post #42

ggnore7452 wrote: post #40

I’ve done a similar PDF → Markdown workflow. For each page: - Extract text as usual. - Capture the whole page as an image (~200 DPI). - Optionally extract images/graphs within the page and include them in the same LLM call. - Optionally add a bit of context from neighboring pages. Then wrap everything with a clear prompt (structured output + how you want graphs handled), and you’re set. At this point, models like GPT…

>are cheap and strong enough to make this practical. It all depends on the scale you need them, with the API it's easy to generate millions of tokens without thinking.

You don't need full reasoning to get accurate results, so even with GPT5 it's still pretty cheap for a one-time job and easy to reason about costs. It's certainly cheaper if you have data where reliability is key and classical OCR will undoubtedly require some manual data cleaning...

I can recommend the Mistral OCR API [1] if you have large jobs and don't want to think about it too much.

[1] https://mistral.ai/solutions/document-ai

Re: Llama-Scan: Convert PDFs to Text W Local LLMs

#73

Post by smusamashah » Mon, Aug 18, 2025, 11:31 AM UTC

visarga wrote: post #58

The crucial information is missing - accuracy comparison with other OCR providers. From my experience LLM based OCR might misread the layout and hallucinate values, it is very subtle but sometimes critically wrong. Classical OCR has more precision but doesn't get the layout at all. Combining both has other issues, no approach is 100% reliable.

Any tool that takes a scanned PDF, then overlay's OCRed text over scan so that text becomes searchable?

Re: Llama-Scan: Convert PDFs to Text W Local LLMs

#74

Post by AmazingTurtle » Mon, Aug 18, 2025, 11:31 AM UTC

Yet another Prompt Wrapper

TRANSCRIPTION_PROMPT = """Task: Transcribe the page from the provided book image.

- Reproduce the text exactly as it appears, without adding or omitting anything. - Use Markdown syntax to preserve the original formatting (e.g., headings, bold, italics, lists). - Do not include triple backticks (```) or any other code block markers in your response, unless the page contains code. - Do not include any headers or footers (for example, page numbers). - If the page contains an image, or a diagram, describe it in detail. Enclose the description in an tag. For example:

This is an image of a cat.

"""

Re: Llama-Scan: Convert PDFs to Text W Local LLMs

#75

Post by edwinjones » Mon, Aug 18, 2025, 11:41 AM UTC

AmazingTurtle wrote: post #74

Yet another Prompt Wrapper TRANSCRIPTION_PROMPT = """Task: Transcribe the page from the provided book image. - Reproduce the text exactly as it appears, without adding or omitting anything. - Use Markdown syntax to preserve the original formatting (e.g., headings, bold, italics, lists). - Do not include triple backticks (```) or any other code block markers in your response, unless the page contains code. - Do not in…

Agreed, it really looks like quite a small prompt wrapper: https://github.com/ngafar/llama-scan/blob/main/llama_scan/co...

The url to connect to ollama seems to just be hard coded so I don't see why you couldn't point this at a different machine on your network rather than having Ollama running locally on every machine you need this for like the readme implies.

Re: Llama-Scan: Convert PDFs to Text W Local LLMs

#76

Post by fforflo » Mon, Aug 18, 2025, 12:32 PM UTC

If you're interested in this sort of thing with an SQL flavor, you may find the pgpdf PostgreSQL extension useful https://github.com/Florents-Tselai/pgpdf .

It's basically an SQL wrapper around poppler.

Re: Llama-Scan: Convert PDFs to Text W Local LLMs

#77

Post by Xmd5a » Mon, Aug 18, 2025, 1:01 PM UTC

smusamashah wrote: post #73

visarga wrote: post #58

The crucial information is missing - accuracy comparison with other OCR providers. From my experience LLM based OCR might misread the layout and hallucinate values, it is very subtle but sometimes critically wrong. Classical OCR has more precision but doesn't get the layout at all. Combining both has other issues, no approach is 100% reliable.

Any tool that takes a scanned PDF, then overlay's OCRed text over scan so that text becomes searchable?

https://github.com/ocrmypdf/OCRmyPDF

>OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched

I ... I nailed it.

Re: Llama-Scan: Convert PDFs to Text W Local LLMs

#78

Post by quibono » Mon, Aug 18, 2025, 1:24 PM UTC

Xmd5a wrote: post #77

smusamashah wrote: post #73

Earlier quoted context omitted.

Any tool that takes a scanned PDF, then overlay's OCRed text over scan so that text becomes searchable?

https://github.com/ocrmypdf/OCRmyPDF >OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched I ... I nailed it.

Just a note that OCRmyPDF currently uses Tesseract

Re: Llama-Scan: Convert PDFs to Text W Local LLMs

#79

Post by zarzavat » Mon, Aug 18, 2025, 1:56 PM UTC

agentcoops wrote: post #69

visarga wrote: post #58

The crucial information is missing - accuracy comparison with other OCR providers. From my experience LLM based OCR might misread the layout and hallucinate values, it is very subtle but sometimes critically wrong. Classical OCR has more precision but doesn't get the layout at all. Combining both has other issues, no approach is 100% reliable.

Have you evaluated this lately? Last year or even just earlier this year I would have mostly agreed with you. At this point, however, with at least the documents I have been working on, OCR reliability with GPT5 or Mistral OCR [1] has been much better than even domain-trained classical OCR. If the documents have even slightly complex layout (to say nothing of page numbers or page headings or an uncommon font), the ac…

I asked GPT-5 to OCR a table for me the other day, it hallucinated perhaps 10% of the values. This was a screenshot of a spreadsheet, with large font, not challenging except for the layout.

What's interesting is that I asked it to also read the background colors of the cells and it did much worse on that task.

I believe these models could be useful for a first pass if you are willing to manually review everything they output, but the failure mode is unsettling.

Re: Llama-Scan: Convert PDFs to Text W Local LLMs

#80

Post by Someone » Mon, Aug 18, 2025, 2:56 PM UTC

skygazer wrote: post #49

privatelypublic wrote: post #38

Earlier quoted context omitted.

If you think 1990's ocr- even 2000's OCR is remotely as good as modern OCR... I`v3 g0ta bnedge to sell.

I had an on-screen OCR app on my Amiga in the early 90s that was amazing, so long as the captured text image used a system font. Avoiding all the mess of reality like optics, perspective, sensors and physics and it could be basically perfect.

MacPaint had that in 1983, but it never shipped because Bill Atkinson “was afraid that if he left it in, people would actually use it a lot, and MacPaint would be regarded as an inadequate word processor instead of a great drawing program” (https://www.folklore.org/MacPaint_Evolution.html)

Also shows a way to do that fast:

“ First, he wrote assembly language routines to isolate the bounding box of each character in the selected range. Then he computed a checksum of the pixels within each bounding box, and compared them to a pre-computed table that was made for each known font, only having to perform the full, detailed comparison if the checksum matched.”