Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama-OCR: Document to Markdown

llamaocr.com

Post reply on HN
Print view View original

81–90 of 104 posts

Re: Llama-OCR: Document to Markdown

#81

Post by pbhjpbhj »

danvk wrote: post #80
notsylver wrote: post #17

I've been doing a lot of OCR recently, mostly digitising text from family photos. Normal OCR models are terrible at it, LLMs do far better. Gemini Flash came out on top from the models I tested and it wasn't even close. It still had enough failures and hallucinations to make it faster to write it in by hand. Annoying considering how close it feels to working. This seems worse. Sometimes it replies with just the text,…

I've had really good luck recently running OCR over a corpus of images using gpt-4o. The most important thing I realized was that non-fancy data prep is still important, even with fancy LLMs. Cropping my images to just the text (excluding any borders) and increasing the contrast of the image helped enormously. (I wrote about this in 2015 and this post still holds up well with GPT: https://www.danvk.org/2015/01/07/fin…

Have you tried doing a verification pass: so giving gpt-4o the output of the first pass, and the image, and asking if they can correct the text (or if they match, or...)?

Just curious whether repetition increases accuracy or of it hurt increases the opportunities for hallucinations?

Re: Llama-OCR: Document to Markdown

#83

Post by pbhjpbhj »

famouswaffles wrote: post #18
notsylver wrote: post #17

I've been doing a lot of OCR recently, mostly digitising text from family photos. Normal OCR models are terrible at it, LLMs do far better. Gemini Flash came out on top from the models I tested and it wasn't even close. It still had enough failures and hallucinations to make it faster to write it in by hand. Annoying considering how close it feels to working. This seems worse. Sometimes it replies with just the text,…

>Normal OCR models are terrible at it, LLMs do far better. Gemini Flash came out on top from the models I tested and it wasn't even close. For Normal models, the state of Open Source OCR is pretty terrible. Unfortunately, the closed options from Microsoft, Google etc are much better. Did you try those ? Interesting about Flash, what LLMs did you test ?

The OCR in OneNote is incredible IME. But, I've not tested in a wide range of fonts -- only that I have abysmal handwriting and it will find words that are almost unrecognisable.

Re: Llama-OCR: Document to Markdown

#84

Post by ggerules »

mg wrote: post #34

I gave it a sentence, which I created by placing 500 circles via a genetic algorithm to form a sentence. And then drew with an actual physical circle: https://www.instagram.com/marekgibney/p/BiFNyYBhvGr/ Interestingly, it sees the circles just fine, but not the sentence. It replied with this: The image contains no text or other elements that can be represented in Markdown. It is a visual composition of circles and do…

Was the original LLM ever trained on original material like this?

Pretty cool use of genetic algorithm! Would love to see the code or at least the reward function.

Re: Llama-OCR: Document to Markdown

#85

Post by pbhjpbhj »

thih9 wrote: post #43
DandyDev wrote: post #37

Earlier quoted context omitted.

I can't read this either. Edit: at a distance it's easier to read

If you squint it’s easier too. I wonder if lowering the resolution of the image would make the text visible to ocr.

I wonder if you could do a composite image, like bracketed images, and so give the model multiple goes, for which it could amalgamate results. So, you could do an exposure bracket, do a focus/blur, maybe a stretch/compression, or an adjustment for font-height as a proportion of the image.

Feed all of the alternatives to the model, tell it they each have the same textual content?

Re: Llama-OCR: Document to Markdown

#86

Post by zainia »

nh2 wrote: post #46
nutlope wrote: post #22

Hi all, I'm the author of llama-ocr. Thank you for sharing & for the kind comments! I built this earlier this week since I wanted a simple API to do OCR – it uses llama 3.2 vision (hosted on together.ai, where i work) to parse images into structured markdown. I also have it available as an npm package. Planning to add a bunch of other features like the ability to parse PDFs, output a response in JSON, ect... If anyon…

I put in a bill that has 3 identical line items and it didn't include them as 3 bullet points as usual, but generated a table with a "quantity" column that doesn't exist on the original paper. Is this amount of larger transformation expected/desirable? (It also means that the output is sometimes a bullet point list, sometimes a table, making further automatic processing a bit harder.)

Here's the prompt being used, tweaking that might help: https://github.com/Nutlope/llama-ocr/blob/main/src/index.ts#...

Re: Llama-OCR: Document to Markdown

#89

Post by rasz »

Old scan of Asus P3B-F motherboard schematic from 1997.

- only managed to extract some of the text from Title Block (project name, date etc)

- despite distinct font got all 8/B and 1/I mixed up.

- the actual useful info got turned into

    Tables
    Table 1: [Insert table 1 here]

    Other Elements
    [Insert other elements here]
Post reply on HN