Earlier quoted context omitted.
Well I sure do feel different than a fire ant.
anttention is all we have
Not true, I also have a nice cigar waiting for the rain to go away.
PDF to Text, a challenging problem
91–100 of 206 posts
Re: PDF to Text, a challenging problem
#92So many of these problems have been solved by mozilla pdf.js together with its viewer implementation: https://mozilla.github.io/pdf.js/ .
Any sense on how PDF.js compares against other tools such as pdfminer?
I did some very broad testing of several PDF text extraction tools recently, and PDF.js was one of the slowest.
My use-case was specifically testing their performance as command-line tools, so that will skew the results to an extent. For example, PDFBox was very slow because you're paying the JVM startup cost with each invocation.
Poppler's pdftotext utility and pdfminer.six were generally the fastest. Both produced serviceable plain-text versions of the PDFs, with minor differences in where they placed paragraph breaks.
I also wrote a small program which extracted text using Chrome's PDFium, which also performed well, but building that project can be a nightmare unless you're Google. IBM's Docling project, which uses ML models, produced by far the best formatting, preserving much of the document's original structure – but it was, of course, enormously slower and more energy-hungry.
Disclaimer: I was testing specific PDF files that are representative of the kind of documents my software produces.
Re: PDF to Text, a challenging problem
#93Recently tested a (non-english) pdf ocr with Gemini 2.5 Pro.
First, directly ask it to extract text from pdf. Result: random text blob, not useable.
Second, I converted pdf into pages of jpg. Gemini performed exceptional. Near perfect text extraction with intact format in markdown.
Maybe there's internal difference when processing pdf vs jpg inside the model.
Re: PDF to Text, a challenging problem
#94Recently tested a (non-english) pdf ocr with Gemini 2.5 Pro. First, directly ask it to extract text from pdf. Result: random text blob, not useable. Second, I converted pdf into pages of jpg. Gemini performed exceptional. Near perfect text extraction with intact format in markdown. Maybe there's internal difference when processing pdf vs jpg inside the model.
Model isn’t rendering the PDF probably, just looking in the file for text.
Re: PDF to Text, a challenging problem
#95I've worked on this in my day job: extracting _all_ relevant information from a financial services PDF for a bert based search engine.
The only way to solve that is with a segmentation model followed by a regular OCR model and whatever other specialized models you need to extract other types of data. VLM aren't ready for prime time and won't be for a decade on more.
What worked was using doclaynet trained YOLO models to get the areas of the document that were text, images, tables or formulas: https://github.com/DS4SD/DocLayNet if you don't care about anything but text you can feed the results into tesseract directly (but for the love of god read the manual). Congratulations, you're done.
Here's some pre-trained models that work OK out of the box: https://github.com/ppaanngggg/yolo-doclaynet I found that we needed to increase the resolution from ~700px to ~2100px horizontal for financial data segmentation.
VLMs on the other hand still choke on long text and hallucinate unpredictably. Worse they can't understand nested data. If you give _any_ current model nothing harder than three nested rectangles with text under each they will not extract the text correctly. Given that nested rectangles describes every table no VLM can currently extract data from anything but the most straightforward of tables. But it will happily lie to you that it did - after all a mining company should own a dozen bulldozers right? And if they each cost $35.000 it must be an amazing deal they got, right?
Re: PDF to Text, a challenging problem
#96Some of the unsung heroes of the modern age are the programmers who, through what must have involved a lot of weeping and gnashing of teeth, have managed to implement the find, select, and copy operations in PDF readers.
Re: PDF to Text, a challenging problem
#97Have any of you ever thought to yourself, this is new and interesting, and then vaguely remembered that you spent months or years becoming an expert at it earlier in life but entirely forgot it? And in fact large chunks of the very interesting things you've done just completely flew out of your mind long ago, to the point where you feel absolutely new at life, like you've accomplished relatively nothing, until someth…
Back in... 2006ish? I got annoyed with being unable to copy text from multicolumn scientific papers on my iRex (an early ereader that was somewhat hackable) so dug a bit into why that was. Under the hood, the pdf reader used poppler, so I modified poppler to infer reading order in multicolumn documents using algorithms that tessaract's author (Thomas Breuel) had published for OCR.
It was a bit of a heuristic hack; it was 20 years ago but as I recall poppler's ancient API didn't really represent text runs in a way you'd want for an accessibility API. A version of the multicolumn select made it in but it was a pain to try to persuade poppler's maintainer that subsequent suggestions to improve performance were ok - because they used slightly different heuristics so had different text selections in some circumstances. There was no 'right' answer, so wanting the results to match didn't make sense.
And that's how kpdf got multicolumn select, of a sort.
Using tessaract directly for this has probably made more sense for some years now.
Re: PDF to Text, a challenging problem
#98"PDF to Text" is a bit simplified IMO. There's actually a few class of problems within this category: 1. reliable OCR from documents (to index for search, feed into a vector DB, etc) 2. structured data extraction (pull out targeted values) 3. end-to-end document pipelines (e.g. automate mortgage applications) Marginalia needs to solve problem #1 (OCR), which is luckily getting commoditized by the day thanks to models…
>replace their OCR pipelines with Flash for a fraction of the cost of previous solutions, it's really quite remarkable.
As someone who had to build custom tools because VLMs are so unreliable: anyone that uses VLMs for unprocessed images is in for more pain than all the providers which let LLMs without guard rails interact directly with consumers.
They are very good at image labeling. They are ok at very simple documents, e.g. single column text, centered single level of headings, one image or table per page, etc. (which is what all the MVP demos show). They need another trillion parameters to become bad at complex documents with tables and images.
Right now they hallucinate so badly that you simply _can't_ use them for something as simple as a table with a heading at the top, data in the middle and a summary at the bottom.
Re: PDF to Text, a challenging problem
#99Good old https://linux.die.net/man/1/pdftotext and a little Python on top of its output will get you a long way if your documents are not too crazy. I use it to parse all my bank statements into an sqlite database for analysis.
Re: PDF to Text, a challenging problem
#100One thing I wish someone would write is something like the browser's developer tools ("inspect elements") for PDF — it would be great to be able to "view source" a PDF's content streams (the BT … ET operators that enclose text, each Tj operator for setting down text in the currently chosen font, etc), to see how every “pixel” of the PDF is being specified/generated. I know this goes against the current trend / state-…
Try with cpdf (disclaimer, wrote it): cpdf -output-json -output-json-parse-content-streams in.pdf -o out.json Then you can play around with the JSON, and turn it back to PDF with cpdf -j out.json -o out.pdf No live back-and-forth though.
This might actually be something very valuable to me.
I have a bunch of documents right now that are annual statutory and financial disclosures of a large institute, and they are just barely differently organized from each year to the next to make it too tedious to cross compare them manually. I've been looking around for a tool that could break out the content and let me reorder it so that the same section is on the same page for every report.
This might be it.