There is also PDF to HTML, PDF to Text, MuPDF also has PDF to XML, both projects along with a bucketful of other PDF toolkits have PDF to PS, and there is many many XML, HTML, and Text outputs for PS. Rastering and OCR'ing PDF is like using regex to parse XHTML. My eyes are starting to bleed out, I am done here.
It looks like you make a lot of valid points, but also have an extremely visceral reaction because theres a company out there thats using AI in a way that offends you. I mean fair still. But im a guy who's in the market for a pdf parser service, im happy to pay pretty penny per page processed. I just want a service that works without me thinking for a second about any of the problems you guys are all discussing. What…
It is a hard problem, yes, but you don't solve it by rastering it, OCR, and then using AI. You render it into a structured format. Then at least you don't have to worry about hallucinations, fancy fonts OCR problems, text shaping problems, huge waste of GPU and CPU to paint an image only to OCR it and throw it away.
Use a solution that renders PDF into structured data if you want correct and reliable data.
How do you render into structured data, from disparate letters that are not structured? D10 E1 H0 L2,3,9 O4,7 R8 W6 I'm sure that you could look at that and figure out how to structure it. But I highly doubt that you have a general-purpose computer program that can parse that into structured data, having never encountered such a format before. Yet, that is how many real-world PDF files are composed.
It is called rendering. MuPDF, Poppler, PDFjs, and so on. The problem is that you and everyone else thinks "rendering" means bitmaps. That is not how it works.
Then I would very much appreciate if you would enlighten me. I'm serious, I would love nothing more than for you to prove your point, teach me something, and win an internet argument. Once rendered, do any of the rendering engines have e.g. a selectable or accessible text? Poppler didn't, neither did some Java library that I tried.
For me, learning something new is very much worth losing the internet argument!
Seems like a fairly reasonable decision given all the high quality implementations out there.
How is it reasonable to render the PDF, rasterize it, OCR it, use AI, instead of just using the "quality implementation" to actually get structured data out? Sounds like "I don't know programming, so I will just use AI".
Because PDF is as much a vector graphics format as a document format, you cannot expect the data to be reasonably structured. For example applications can convert text to vector outlines or bitmaps for practical or artistic purposes (anyone who ever had to deal with transparency "flattening" issues knows the pain), ideally they also encode the text in a seperate semantic representation. But many times PDF files are exported from "image centric" programs with image centric workflows (e.g. Illustrator, CorelDraw, Indesign, QuarkXpress, etc) where the main issue being solved for is presentational content, not semantic. For example if I receive a Word document and need to layout it so it fits into my multi column magazine layout I will take the source text and break it into seperate sections which then manually get copy and pasted into InDesign. You can import the document directly but for all kinds of practical reasons this is not the default way of working. Some asides and lists might be broken out of the original flow of text and placed in their own text field, etc. So now you lost the original semantic structure. Remember, this is how desktop publishing evolved: for print, which has no notion of structure or metadata embedded into the ink or paper. Another common usecase is to simply have resolution independent graphics, again, display purposes only, no structured data is required nor expected.
We tried the xml structured route, only to end up with pea soup afterwards. Rasterizing and OCR was the only way to get standardized output.
I know OCR is easier to set up, but you lose a lot going that way. We process several million pages from Newspapers and Magazines from all over the world with medium to very high complexity layouts. We built the PDF parser on top of open source PDF libraries, and this gives many advantages: • We can accurately get headlines other text placed on top on images. OCR is generally hopeless with text placed on top of image…
> To do it, we completely abandon the PDF text-structure and only use the individual location of each letter. Then we combine letter positions to words, words to lines, and lines to text-blocks using a number of algorithms.
We use the structure blocks that we generated with machine learning afterwards, so this is just the first step in analyzing the page.
Do you happen to have any sources for learning more about the piecing together process? E.g. the overal process and the algorithms involved etc. It sounds like an interesting problem to solve.
What's the economic justification? Let's assume we have a staff of 10 and they're fully allocated to committed features and deadlines, so they can't be shifted elsewhere. You're the CTO and you ask the BOD for another $150k/y (fully burdened) + equity to hire a new developer with PDF skills. The COB asks you directly: "You can get a battle-tested PDF parser off-the-shelf for little or no cost. We're not in the PDF pa…
The mistake all of you're making is the assumption that PDF rendering means rasteration. Everything else crumbles down from that misconception.
So if you receive a pdf full of sections containing prerasterized text (e.g adverts, 3d rendered text with image effects, scanned documents, handwritten errata), what do you do? You cannot use OCR because apparently only pdf-illiterate idiots would try such a thing?
Sometimes the characters aren’t even characters, just paths
Wouldn't that be very space inefficient to repeat the paths every time a letter appears in the file? Or you mean that glyph Ids don't necessarily map to Unicode?
Outlines are just a practical way of handling less common display cases.
Just to give a practical example. Imagine a Star Wars advert that has the Star Wars logo at the top, specified in outlines because that's what every vector logo uses. Below it the typical Star Wars intro text stretched into perspective, also using outlines, because that's the easiest (display engine doesn't need complicated transformation stack), efficient to render (you have to render the outlines anyway), and most robust way (looks the same everywhere), way of implementing transformations in text. You also don't have to supply the font file, which comes with licensing issues, etc. Also whenever compositing and transparency are involved, with color space conversion nonsense, it's more robust to "bake" the effect via constructive geometry operations, etc, to prevent display issues on other devices, which are surprisingly common.
It is called rendering. MuPDF, Poppler, PDFjs, and so on. The problem is that you and everyone else thinks "rendering" means bitmaps. That is not how it works.
Then I would very much appreciate if you would enlighten me. I'm serious, I would love nothing more than for you to prove your point, teach me something, and win an internet argument. Once rendered, do any of the rendering engines have e.g. a selectable or accessible text? Poppler didn't, neither did some Java library that I tried. For me, learning something new is very much worth losing the internet argument!
I have explained the details in other comments, have a look. But you can start by looking at pdftotext from Poppler, it is ready to go for 60-70% of cases with -layout flag, with bbox-layout you get even more details.
So you parse PDFs, but also OCR images, to somehow get better results? Do you know you could just use the parsing engine that renders the PDF to get the output? I mean, why raster it, OCR it, and then use AI? Sounds creating a problem to use AI to solve it.
Well, you clearly hasn't parsed a wide variety of pdfs. Because if you had, you had been exposed to pdfs that contain only images, or those that contain embedded text, but that embedded text is utter nonsense and doesn't match what is shown on the page when rendered. And that is before we even get into text structure, because as everyone knows, reading text is easier if things like paragraphs, columns and tables are…
If your rendering engine doesn't output what is shown, your engine is broken, and it can be broken whatever you render it into bitmap or structured data.
But all those problems exist when rendering into a surface or rastering. I just don't understand how one thinks, this is a hard problem, let me make it harder by solving the problem into another kind of problem that is just as hard as solving it in the first place (PDF to structured data vs PDF to raster). And then solve the new problem, which is also hard. It is absurd.
Sometimes scanned documents are structured really weird, especially for tables. Visually, we can recognize the intention when it's rendered, and so can the AI, but you practically have to render it to recover the spatial context.
The mistake all of you're making is the assumption that PDF rendering means rasteration. Everything else crumbles down from that misconception.
So if you receive a pdf full of sections containing prerasterized text (e.g adverts, 3d rendered text with image effects, scanned documents, handwritten errata), what do you do? You cannot use OCR because apparently only pdf-illiterate idiots would try such a thing?
I wouldn't start by rastering the rest of the PDF. In business world, unlike academia and bootleg books and file sharing, majority of PDFs are computer generated. I know because I do this for a living.