Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama-OCR: Document to Markdown

llamaocr.com

Post reply on HN
Print view View original

91–100 of 104 posts

Re: Llama-OCR: Document to Markdown

#91

Post by gcr »

nutlope wrote: post #22

Hi all, I'm the author of llama-ocr. Thank you for sharing & for the kind comments! I built this earlier this week since I wanted a simple API to do OCR – it uses llama 3.2 vision (hosted on together.ai, where i work) to parse images into structured markdown. I also have it available as an npm package. Planning to add a bunch of other features like the ability to parse PDFs, output a response in JSON, ect... If anyon…

How accurate is this?

When compared with existing OCR systems, what sorts of mistakes does it make?

Re: Llama-OCR: Document to Markdown

#92

Post by rch »

nutlope wrote: post #22

Hi all, I'm the author of llama-ocr. Thank you for sharing & for the kind comments! I built this earlier this week since I wanted a simple API to do OCR – it uses llama 3.2 vision (hosted on together.ai, where i work) to parse images into structured markdown. I also have it available as an npm package. Planning to add a bunch of other features like the ability to parse PDFs, output a response in JSON, ect... If anyon…

I've had trouble with pulling scientific content out of poster PDFs, mostly because e.g. nougat falls apart with different layouts.

Have you considered that usage yet?

Re: Llama-OCR: Document to Markdown

#93

Post by 8n4vidtmkvmk »

bosie wrote: post #32
8n4vidtmkvmk wrote: post #21

Earlier quoted context omitted.

That's a bummer. I'm trying to do the exact same thing right now, digitize family photos. Some of mine have German on the back. The last OCR to hit headlines was terrible, was hoping this would be better. ChatGPT 4o has been good though, when I paste individual images into the chat. I haven't tried with the API yet, not sure how much that would cost me to process 6500 photos, many of which are blank but I don't have…

Use a local rubbish model to extract text. If it doesn’t find any on the back, don’t send it to chatgtp? Terrascan comes to mind

"Terrascan" is a vision model? The only hits I'm getting are for a static code analyzer.

Re: Llama-OCR: Document to Markdown

#94

Post by sinuhe69 »

Very funny. I put in 3 screen captures of a (long) document, and it did relatively well. But when I proof-read it, I realized the AI has made up passages that were not there!

The reason is probably due to the nature of screen capturing, some sentences or paragraphs were cut short. That probably kicked off the “fill in the blank” nature of the LLM and it could not resist to leave these paragraphs stand unfinished :LOL. It even put in a short conclusion paragraph that was not in the original document at all!

Re: Llama-OCR: Document to Markdown

#95

Post by abenga »

sinuhe69 wrote: post #94

Very funny. I put in 3 screen captures of a (long) document, and it did relatively well. But when I proof-read it, I realized the AI has made up passages that were not there! The reason is probably due to the nature of screen capturing, some sentences or paragraphs were cut short. That probably kicked off the “fill in the blank” nature of the LLM and it could not resist to leave these paragraphs stand unfinished :LOL…

It boggles my mind that a technology where "making things up" is even a remote possibility is ever actually considered for use in the real world.

Re: Llama-OCR: Document to Markdown

#96

Post by AmazingTurtle »

cess11 wrote: post #73
AmazingTurtle wrote: post #44

One can combine apache tika OCR and feed it together with the image into LLM to fix typos.

While I'm a fan of Tika a lot of people get queasy from Java and XML, they might be better served by their preferred scripting language and https://github.com/ocrmypdf/OCRmyPDF , which has the same OCR engine.

May I introduce you to `apache/tika:2.9.2.1-full` with a REST API on 9998.

Re: Llama-OCR: Document to Markdown

#97

Post by cess11 »

AmazingTurtle wrote: post #96
cess11 wrote: post #73

Earlier quoted context omitted.

While I'm a fan of Tika a lot of people get queasy from Java and XML, they might be better served by their preferred scripting language and https://github.com/ocrmypdf/OCRmyPDF , which has the same OCR engine.

May I introduce you to `apache/tika:2.9.2.1-full` with a REST API on 9998.

Not sure what you mean. Are they making Graal-builds you can run standalone now? I only use Tika through Maven at work, might not be up to date on what happens in the project.

Re: Llama-OCR: Document to Markdown

#98

Post by timmattison »

nirav72 wrote: post #55
Eisenstein wrote: post #52

Earlier quoted context omitted.

I made a script which does exactly the same thing but locally using koboldcpp for inference. It downloads MiniCPM-V 2.6 with image projector the first time you run it. If you want to use a different model you can, but you will want to edit the instruct template to match. * https://github.com/jabberjabberjabber/LLMOCR

MiniCPM-v 2.6 is probably the best self-hosted vision model I have used so far. Not just for OCR, but also image analysis. I have it setup, so my NVR (frigate) sends couple of images upon motion alert from a driveway security camera to Ollama with minicpm-v 2.6. I’m able to get a reasonably accurate description of the vehicle that pulled into the driveway. Including describing the person that exits the vehicle and al…

I love this. Can you share the source?

Re: Llama-OCR: Document to Markdown

#99

Post by danvk »

pbhjpbhj wrote: post #81
danvk wrote: post #80

Earlier quoted context omitted.

I've had really good luck recently running OCR over a corpus of images using gpt-4o. The most important thing I realized was that non-fancy data prep is still important, even with fancy LLMs. Cropping my images to just the text (excluding any borders) and increasing the contrast of the image helped enormously. (I wrote about this in 2015 and this post still holds up well with GPT: https://www.danvk.org/2015/01/07/fin…

Have you tried doing a verification pass: so giving gpt-4o the output of the first pass, and the image, and asking if they can correct the text (or if they match, or...)? Just curious whether repetition increases accuracy or of it hurt increases the opportunities for hallucinations?

I have not, but that's a great idea!

Re: Llama-OCR: Document to Markdown

#100

Post by wriggler »

philips wrote: post #27

I have recently used llama3.2-vision to handle some paper bidsheets for a charity auction and it is fairly accurate with some terrible handwriting. I hope to use it for my event next year. I do find it rather annoying not being able to get it to consistently output a CSV though. ChatGPT and Gemini seem better at doing that but I haven’t tried to automate it. The scale of my problem is about 100 pages of bidsheets and…

I'd love to hear how Handwriting OCR (https://www.handwritingocr.com) compares for your task.

It's not free, but its accuracy for for handwritten documents is the best out there (I am the founder, so am biased, but I'm really excited about where the accuracy is now). It could save you time and for your 100 page project would cost only $12.

Post reply on HN