Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Marker: Convert PDF to Markdown quickly with high accuracy

github.com

Post reply on HN
Print view View original

11–20 of 101 posts

Re: Marker: Convert PDF to Markdown quickly with high accuracy

#11

Post by defsectec »

alsodumb wrote: post #4

Great work! I am a bit confused with the comparison with nougat throughout the repo. Nougat was specifically trained for academic documents, and I don't think anyone ever claimed Nougat was the best OCR model out there. That's kinda clear in your benchmark too where you mention that nougat has higher accuracy on arxiv documents. You also mention that marker will convert fewer equations when compared to nougat, and ye…

How do you think nougat would handle RPG rulebook PDFs?

I'm looking for a food OCR model to help me transcribe sections of RPG books to markdown. Ideally, I'd like the emphasis such as bold or italics to be transcribed.

The combo of text, numbers, and math symbols seems similar to technical and academic writing, but often has weird formatting, text boxes in the margins, and many diagrams.

Re: Marker: Convert PDF to Markdown quickly with high accuracy

#12

Post by fshr »

alsodumb wrote: post #4

Great work! I am a bit confused with the comparison with nougat throughout the repo. Nougat was specifically trained for academic documents, and I don't think anyone ever claimed Nougat was the best OCR model out there. That's kinda clear in your benchmark too where you mention that nougat has higher accuracy on arxiv documents. You also mention that marker will convert fewer equations when compared to nougat, and ye…

> I don't think anyone ever claimed Nougat was the best OCR model out there

Comparing two things doesn't inherently imply the previous thing was touted about with superlatives. It's just a way to juxtapose the new thing with something that may be familiar. As you said, nougat is easy to install/run so it makes sense they'd compare it. Would it be better if they could add more libraries in the comparison? Absolutely; that'd be helpful.

Re: Marker: Convert PDF to Markdown quickly with high accuracy

#14

Post by crooked-v »

defsectec wrote: post #8

This looks amazing, I'll have to play around with this over the weekend. I regularly hand transcribe RPG PDFs scans from dubious sources that have not always been run through OCR to have selectable text. If it has, it wasn't always done very well. It's literally faster to type it all myself than fix all the errors from copy-pasting (or after using OCR to turn it into text). Even if the file was an official PDF the fo…

> I regularly hand transcribe RPG PDFs scans from dubious sources

Heh, that was my immediate thought too. There's a ton of RPG stuff that never had any kind of physical release and is totally orphaned as IP.

Re: Marker: Convert PDF to Markdown quickly with high accuracy

#16

Post by alsodumb »

defsectec wrote: post #11
alsodumb wrote: post #4

Great work! I am a bit confused with the comparison with nougat throughout the repo. Nougat was specifically trained for academic documents, and I don't think anyone ever claimed Nougat was the best OCR model out there. That's kinda clear in your benchmark too where you mention that nougat has higher accuracy on arxiv documents. You also mention that marker will convert fewer equations when compared to nougat, and ye…

How do you think nougat would handle RPG rulebook PDFs? I'm looking for a food OCR model to help me transcribe sections of RPG books to markdown. Ideally, I'd like the emphasis such as bold or italics to be transcribed. The combo of text, numbers, and math symbols seems similar to technical and academic writing, but often has weird formatting, text boxes in the margins, and many diagrams.

I'm not completely sure to be honest, but you should try it yourself with a sample page! I believe hugging face hosts it online on their demo pages so you don't even have to install the package to test on one page.

Re: Marker: Convert PDF to Markdown quickly with high accuracy

#18

Post by prmoustache »

lgats wrote: post #13

It'd be really great if there was something like this that also supported image extraction

Especially for those that want to move out of Confluence. It is rather easy to obtain a docx or pdf from the API as well as the raw, uncompressed attachements, a bit more complicated to convert said files to markdown with full quality attachements and no formatting errors on every pages.

Re: Marker: Convert PDF to Markdown quickly with high accuracy

#19

Post by pk-protect-ai »

phi-go wrote: post #17
nicodjimenez wrote: post #9

[flagged]

You should mention that you are the CEO of Mathpix.

It actually doesn't matter. For my cases, I found Mathpix to be much more reliable than Nougat, for example. So, when you have hundreds of documents to convert a year and little time for manual labor on the results, paying a yearly "pro" subscription fee is worth it. However, it will really hit your pocket when you need to prepare datasets from thousands of PDFs... That's what you can't afford without a budget allocation from your project.

Re: Marker: Convert PDF to Markdown quickly with high accuracy

#20

Post by danofsteel32 »

I have an odd usecase that I've yet to find a good solution to: Reading construction documents (Blueprints are always PDF). I've had much better luck parsing DXF (AutoCAD) files but it's not always easy to get an architect to send them to me even if I'm the GC on the job.
Post reply on HN