[dead]
PDF to Text, a challenging problem
141–150 of 206 posts
Re: PDF to Text, a challenging problem
#142Earlier quoted context omitted.
It can also have embedded binary streams. It was not made for text. It was made for layout and graphics. You give nice examples, but each of those lines could have been broken up into one call per character, or per word, even out of order.
It can also use fonts which map glyphs via characters which do not represent the final visual item e.g. "PDF" could be "1#F" and you only really know what it looks like by rendering then viewing/OCR. A nice file won't, but sometimes the best work is in not dealing with nice things.
See this is why we can't have nice things.
Re: PDF to Text, a challenging problem
#143This is mostly what I worked on for many years at Apple with reasonable success. The main secret was to accept that everything was geometry, and use cluster analysis to try to distinguish between word gaps and letter gaps. On many PDF documents, it works really well, but there are so many different kinds of PDF documents that there are always cases were the results are not that great. If I were to do it today, I would stick with geometry, avoid OCR completely, but use machine learning. One big advantage for machine learning is that I could use existing tools to generate PDFs from known text, so that the training phase could be completly automatic.
(Here is Bertrand Serlet announcing the feature at WWDC in 2009: https://youtu.be/FTfChHwGFf0?si=wNCfI9wZj1aj9rY7&t=308)
Re: PDF to Text, a challenging problem
#144Earlier quoted context omitted.
We will never get back the collective man-decades of time that has been burned by this format. When will the madness stop?
When we get an alternative that can: (1) be stored in a single file (2) Allow tables, images and anything else that can be shown on a piece paper (3) Won't have animation, fold-out text, or anything that cannot be be shown on a piece of paper (4) won't require Javascript or access to external sites that means never.. We've got lucky we at least got PDF before "web designers" made (3) impossible, and marketers made (4…
Behold a Bitmap.
But for real, thats a pretty easy set of hurdles. Really the barrier is the psychological fallacy that PDF's are immutable.
Re: PDF to Text, a challenging problem
#145Have any of you ever thought to yourself, this is new and interesting, and then vaguely remembered that you spent months or years becoming an expert at it earlier in life but entirely forgot it? And in fact large chunks of the very interesting things you've done just completely flew out of your mind long ago, to the point where you feel absolutely new at life, like you've accomplished relatively nothing, until someth…
We will never get back the collective man-decades of time that has been burned by this format. When will the madness stop?
The solution always has been in plain sight - just make XML-based format. Nobody liked it, except OpenDocument and eventually Microsoft. Though these formats serve different purpose, new, similar one, could be created with picture-perfect features.
Re: PDF to Text, a challenging problem
#146Earlier quoted context omitted.
When we get an alternative that can: (1) be stored in a single file (2) Allow tables, images and anything else that can be shown on a piece paper (3) Won't have animation, fold-out text, or anything that cannot be be shown on a piece of paper (4) won't require Javascript or access to external sites that means never.. We've got lucky we at least got PDF before "web designers" made (3) impossible, and marketers made (4…
Behold a Bitmap. But for real, thats a pretty easy set of hurdles. Really the barrier is the psychological fallacy that PDF's are immutable.
Should have added "looks good on screen and on paper", "stores text compactly" and "multiple pages supported" :) And yes, that's a pretty easy set of hurdles. I wish we'd standardized on DjVu instead.
Re "PDF's are immutable." - that's not a psychological fallacy, that's a primary advantage of PDFs. If I wanted mutable format, I'd take an odt (or rtf or a doc). "Output only" format allows one to use the very latest version of editor app, while having the result working even in ancient readers, something very desirable in many contexts.
Re: PDF to Text, a challenging problem
#147Earlier quoted context omitted.
When we get an alternative that can: (1) be stored in a single file (2) Allow tables, images and anything else that can be shown on a piece paper (3) Won't have animation, fold-out text, or anything that cannot be be shown on a piece of paper (4) won't require Javascript or access to external sites that means never.. We've got lucky we at least got PDF before "web designers" made (3) impossible, and marketers made (4…
dvi ? https://en.m.wikipedia.org/wiki/Device_independent_file_form...
> For a DVI file to be printed or even properly previewed, the fonts it references must be already installed.
If you want alternatives, I'd choose DjVu. But it's too late now, everyone is converged on PDFs, and the alternatives are not good enough to warrant the switch.
Re: PDF to Text, a challenging problem
#148They should called it NDF - Non-Portable Document Format.
Maybe Printable Document Format.
Re: PDF to Text, a challenging problem
#149Earlier quoted context omitted.
Behold a Bitmap. But for real, thats a pretty easy set of hurdles. Really the barrier is the psychological fallacy that PDF's are immutable.
Should have added "looks good on screen and on paper", "stores text compactly" and "multiple pages supported" :) And yes, that's a pretty easy set of hurdles. I wish we'd standardized on DjVu instead. Re "PDF's are immutable." - that's not a psychological fallacy, that's a primary advantage of PDFs. If I wanted mutable format, I'd take an odt (or rtf or a doc). "Output only" format allows one to use the very latest v…
Word can edit pretty much any pdf these days, the issue is that it will often garble the attempt.
Re: PDF to Text, a challenging problem
#150Below is a PDF. It is a .txt file. I can save it with a .pdf extension and open it in a PDF viewer. I can make changes in a text editor. For example, by editing this text file, I can change the text displayed on the screen when the PDF is opened, the font, font size, line spacing, the maximum characters per line, number of lines per page, the paper width and height, as well as portrait versus landscape mode. %PDF-1.4…
This is the "Hello World" of PDFs.
Most pdfs these days have all of the objs compressed with deflate.
and then, because that didn't make it difficult enough to follow, a lot of pdfs have most of the objects grouped up inside object stream type objects which then get compressed. So you can't have text editor search for a "6 0 Obj" when you are tracking down the end of a "6 0 R"