31–40 of 101 posts
I'm very excited about it.
Let's build a pipeline: all the pdfs -> markdown them all -> archive.org them all
Let's not underestimate the impact of such tool: we are talking about freeing up tons of knowledge from a "good for consumption/bad for distribution" format. I'm very excited about it. Let's build a pipeline: all the pdfs -> markdown them all -> archive.org them all
Even though this would only cover a small part of the needs or use cases, it will still be hugely useful if it works well.
Let's not underestimate the impact of such tool: we are talking about freeing up tons of knowledge from a "good for consumption/bad for distribution" format. I'm very excited about it. Let's build a pipeline: all the pdfs -> markdown them all -> archive.org them all
Yes, there is an enormous interest in this kind of thing, not the least in larger organizations with tons of PDF documents in various forms. Even though this would only cover a small part of the needs or use cases, it will still be hugely useful if it works well.
Earlier quoted context omitted.
Yes, there is an enormous interest in this kind of thing, not the least in larger organizations with tons of PDF documents in various forms. Even though this would only cover a small part of the needs or use cases, it will still be hugely useful if it works well.
cough L cough L cough M cough anyone? :)
This looks amazing, I'll have to play around with this over the weekend. I regularly hand transcribe RPG PDFs scans from dubious sources that have not always been run through OCR to have selectable text. If it has, it wasn't always done very well. It's literally faster to type it all myself than fix all the errors from copy-pasting (or after using OCR to turn it into text). Even if the file was an official PDF the fo…
Earlier quoted context omitted.
cough L cough L cough M cough anyone? :)
Yeah, I know, but a lot of this content can be pretty sensitive, and might not be allowed to upload outside organization networks sometimes (hospitals, governments etc).
It'd be really great if there was something like this that also supported image extraction
My current workflow (for getting a magazine onto a website) is Calibre's HTMLZ export, then through Pandoc to markdown. It produces good enough Markdown to feed in to Hugo, and extracts images. I've been through a number of options in the past and this is what I've settled on.
Great stuff! I have a question regarding the output of Nougat: Where do the "hallucinations" come from (just scroll through the Nougat output of the Think Python example to see what I mean)? Nevermind, i just read it runs it through an LLM, so hallucinations are par for the course.