Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama-OCR: Document to Markdown

llamaocr.com

Post reply on HN
Print view View original

1–10 of 104 posts

Re: Llama-OCR: Document to Markdown

#2

Post by bbor »

Looks awesome! Been doing a lot of OCR recently, and love the addition to the space. The reigning champion in the PDF -> Markdown space (AFAIK) is Facebook's Nougat[1], and I'm excited to hook this up to DSPy and see which works better for philosophy books. This repo links the Zerox[2] project by some startup, which also looks awesome, and certainly more smoothly advertised than Nougat. Would love corrections/advice from any actual experts passing by this comment section :)

That said, I have a few questions if OP/anyone knows the answers:

1. What is Together.ai, and is this model OSS? Their website sells them as a hosting service, and the "Custom Models" page[3] seems to be about custom finetuning, not, like, training new proprietary models in-house. They might have a HuggingFace profile but it's hard to tell if it's them https://huggingface.co/TogetherAI

2. The GitHub says "hosted demo", but the hosting part is just the tiny (clean!) WebGUI, yes? It's implied that this functionality is and will always be available only through API calls?

P.S. The header links are broken on my desktop browser -- no onClick triggered

[1] https://facebookresearch.github.io/nougat/

[2] https://github.com/getomni-ai/zerox

[3] https://www.together.ai/products#custom-models

Re: Llama-OCR: Document to Markdown

#4

Post by magicalhippo »

bbor wrote: post #2

Looks awesome! Been doing a lot of OCR recently, and love the addition to the space. The reigning champion in the PDF -> Markdown space (AFAIK) is Facebook's Nougat[1], and I'm excited to hook this up to DSPy and see which works better for philosophy books. This repo links the Zerox[2] project by some startup, which also looks awesome , and certainly more smoothly advertised than Nougat. Would love corrections/advice…

Yeah was hoping for something I could self-host, both for privacy and cost.

Re: Llama-OCR: Document to Markdown

#7

Post by gexla »

bbor wrote: post #2

Looks awesome! Been doing a lot of OCR recently, and love the addition to the space. The reigning champion in the PDF -> Markdown space (AFAIK) is Facebook's Nougat[1], and I'm excited to hook this up to DSPy and see which works better for philosophy books. This repo links the Zerox[2] project by some startup, which also looks awesome , and certainly more smoothly advertised than Nougat. Would love corrections/advice…

My guess is together.ai is at least partially sponsoring the demo.

Re: Llama-OCR: Document to Markdown

#8

Post by Eisenstein »

All it does is send the image to Llama 3.2 Vision and ask for it to read the text.

Note that this is just as open to hallucination as any other LLM output, because what it is doing is not reading the pixels looking for text characters, but describing the picture, which uses the images it trained on and their captions to determine what the text is. It may completely make up words, especially if it can't read them.

Re: Llama-OCR: Document to Markdown

#9

Post by jurnalanas »

bbor wrote: post #2

Looks awesome! Been doing a lot of OCR recently, and love the addition to the space. The reigning champion in the PDF -> Markdown space (AFAIK) is Facebook's Nougat[1], and I'm excited to hook this up to DSPy and see which works better for philosophy books. This repo links the Zerox[2] project by some startup, which also looks awesome , and certainly more smoothly advertised than Nougat. Would love corrections/advice…

the project author is Devrel from Together.ai. This is a fantastic way to advertise a dev tool, though.
Post reply on HN