Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nanonets-OCR-s – OCR model that transforms documents into structured markdown

huggingface.co

Post reply on HN
Print view View original

51–60 of 85 posts

Re: Nanonets-OCR-s – OCR model that transforms documents into structured markdown

#51

Post by agoose77 »

mgr86 wrote: post #38
esafak wrote: post #35

Earlier quoted context omitted.

First I heard of it. https://en.wikipedia.org/wiki/Text_Encoding_Initiative

Understandable. I work in academic publishing, and while the XML is everywhere crowd is graying, retiring, or even dying :( it still remains an excellent option for document markup. Additionally, a lot of government data produced in the US and EU make heavy use of XML technologies. I imagine they could be an interested consumer of Nanonets-OCR. TEI could be a good choice as well tested and developed conversions exist…

Do check out MyST Markdown (https://mystmd.org)! Academic publishing is a space that MyST is being used, such as https://www.elementalmicroscopy.com/ via Curvenote.

(I'm a MyST contributor)

Re: Nanonets-OCR-s – OCR model that transforms documents into structured markdown

#52

Post by agoose77 »

starkparker wrote: post #47
ks2048 wrote: post #21

It’s a shame all these models target markdown and not something with more structure and a specification. There are different flavors of Markdown and limited support for footnotes, references, figures, etc.

I was more excited to hear about "structured Markdown" than the LLM OCR model, but the extent of it just seems to be tagging certain elements. It's useful in the LLM context but not as much outside of it.

Feel free to check out MyST Markdown, which very much aims to specify "structured Markdown": https://mystmd.org

Re: Nanonets-OCR-s – OCR model that transforms documents into structured markdown

#54

Post by RicoElectrico »

PixelPanda wrote: post #2

Full disclaimer: I work at Nanonets Excited to share Nanonets-OCR-s, a powerful and lightweight (3B) VLM model that converts documents into clean, structured Markdown. This model is trained to understand document structure and content context (like tables, equations, images, plots, watermarks, checkboxes, etc.). Key Features: LaTeX Equation Recognition Converts inline and block-level math into properly formatted LaTe…

Could be it used to (maybe with help of a downstream LLM) parse a photo/PDF of a restaurant menu into a JSON file conforming to a schema? Or would bigger, hosted multimodal LLMs work better in such case?

Re: Nanonets-OCR-s – OCR model that transforms documents into structured markdown

#55

Post by 2pointsomone »

kordlessagain wrote: post #43

I created a Powershell script to run this locally on any PDF: https://gist.github.com/kordless/652234bf0b32b02e39cef32c71e... It does work, but it is very slow on my older GPU (Nvidia 1080 8GB). I would say it's taking at least 5 minutes per page right now, but maybe more. Edit: If anyone is interested in trying a PDF to markdown conversion utility built this that is hosted on Cloud Run (with GPU support), let me kno…

Very very interested!

Re: Nanonets-OCR-s – OCR model that transforms documents into structured markdown

#56

Post by gibsonf1 »

PixelPanda wrote: post #2

Full disclaimer: I work at Nanonets Excited to share Nanonets-OCR-s, a powerful and lightweight (3B) VLM model that converts documents into clean, structured Markdown. This model is trained to understand document structure and content context (like tables, equations, images, plots, watermarks, checkboxes, etc.). Key Features: LaTeX Equation Recognition Converts inline and block-level math into properly formatted LaTe…

Does it hallucinate with the LLM being used?

Re: Nanonets-OCR-s – OCR model that transforms documents into structured markdown

#57

Post by j45 »

generalizations wrote: post #44
PixelPanda wrote: post #2

Full disclaimer: I work at Nanonets Excited to share Nanonets-OCR-s, a powerful and lightweight (3B) VLM model that converts documents into clean, structured Markdown. This model is trained to understand document structure and content context (like tables, equations, images, plots, watermarks, checkboxes, etc.). Key Features: LaTeX Equation Recognition Converts inline and block-level math into properly formatted LaTe…

Does it have a way to extract the images themselves, or is that still a separate process later?

If you are after extracting images from pdfs there’s plenty of tools that do that just fine without LLMs.

Re: Nanonets-OCR-s – OCR model that transforms documents into structured markdown

#58

Post by generalizations »

j45 wrote: post #57
generalizations wrote: post #44

Earlier quoted context omitted.

Does it have a way to extract the images themselves, or is that still a separate process later?

If you are after extracting images from pdfs there’s plenty of tools that do that just fine without LLMs.

I mean, ideally it would be in context, so the generated markdown references the correct image at the correct location in the doc. Unless that's what you're talking about? In which case I don't know about those tools.

Re: Nanonets-OCR-s – OCR model that transforms documents into structured markdown

#59

Post by nattaylor »

gibsonf1 wrote: post #56
PixelPanda wrote: post #2

Full disclaimer: I work at Nanonets Excited to share Nanonets-OCR-s, a powerful and lightweight (3B) VLM model that converts documents into clean, structured Markdown. This model is trained to understand document structure and content context (like tables, equations, images, plots, watermarks, checkboxes, etc.). Key Features: LaTeX Equation Recognition Converts inline and block-level math into properly formatted LaTe…

Does it hallucinate with the LLM being used?

The base model is Qwen2.5-VL-3B and the announcement says a limitation is "Model can suffer from hallucination"

Re: Nanonets-OCR-s – OCR model that transforms documents into structured markdown

#60

Post by viraptor »

agoose77 wrote: post #51
mgr86 wrote: post #38

Earlier quoted context omitted.

Understandable. I work in academic publishing, and while the XML is everywhere crowd is graying, retiring, or even dying :( it still remains an excellent option for document markup. Additionally, a lot of government data produced in the US and EU make heavy use of XML technologies. I imagine they could be an interested consumer of Nanonets-OCR. TEI could be a good choice as well tested and developed conversions exist…

Do check out MyST Markdown ( https://mystmd.org )! Academic publishing is a space that MyST is being used, such as https://www.elementalmicroscopy.com/ via Curvenote. (I'm a MyST contributor)

Do you know why myst got traction, instead of RST which seems to have all the custom tagging and extensibility build in from the beginning?
Post reply on HN