I always think about the invoicing use-case: digital systems should be able to easy extract data from the file while still being formatted visually for humans. It seems like the tech world would be much better off if we migrated to a better format.
PDF to Text, a challenging problem
131–140 of 206 posts
Re: PDF to Text, a challenging problem
#132Earlier quoted context omitted.
That's true, but it's dependent on the creator of the PDF having aligned incentives with the consumer of the PDF. In the e-Discovery field, it's commonplace for those providing evidence to dump it into a PDF purely so that it's harder for the opposing side's lawyers to consume. If both sides have lots of money this isn't a barrier, but for example public defenders don't have funds to hire someone (me!) to process the…
> The solution is to make it illegal to do this: wiretap data, for example, should be provided in a standardized machine-readable format. There's no ethical reason for simple technical friction to be affecting the outcomes of criminal proceedings. I can’t speak to wiretaps specifically, but when it comes to the legal field, this is usually already how it operates. GDPR, for example, makes specific provisions that use…
Re: PDF to Text, a challenging problem
#133Earlier quoted context omitted.
It's still fresh with me, 7 or 8 years ago in my 20s, perhaps you are a bit older? Otherwise wouldn't hurt to do a checkup with a physician
No I'm just good at forgetting things, I've been practicing since I was a kid.
Re: PDF to Text, a challenging problem
#134Below is a PDF. It is a .txt file. I can save it with a .pdf extension and open it in a PDF viewer. I can make changes in a text editor. For example, by editing this text file, I can change the text displayed on the screen when the PDF is opened, the font, font size, line spacing, the maximum characters per line, number of lines per page, the paper width and height, as well as portrait versus landscape mode. %PDF-1.4…
"PDF" is an acronym for for "Portable Document Format" "2.3.2 Portability A PDF file is a 7-bit ASCII file, which means PDF files use only the printable subset of the ASCII character set to describe documents even those with images and special characters. As a result, PDF files are extremely portable across diverse hardware and operating system environments." https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstanda…
Great, so PDF source code is easily printable?
Re: PDF to Text, a challenging problem
#135We[1] Create "Units of Thought" from PDF's and then work with those for further discovery where a "Unit of Thought" is any paragraph, title, note heading - something that stands on its own semantically. We then create a hierarchy of objects from that pdf in the database for search and conceptual search - all at scale. [1] https://graphmetrix.com/trinpod-server https://trinapp.com
I'm tempted to try it. My use case right now is a set of documents which are annual financial and statutory disclosures of a large institution. Every year they are formatted / organized slightly differently which makes it enormously tedious to manually find and compare the same basic section from one year to another, but they are consistent enough to recognize analogous sections from different years due to often reus…
Re: PDF to Text, a challenging problem
#136Have any of you ever thought to yourself, this is new and interesting, and then vaguely remembered that you spent months or years becoming an expert at it earlier in life but entirely forgot it? And in fact large chunks of the very interesting things you've done just completely flew out of your mind long ago, to the point where you feel absolutely new at life, like you've accomplished relatively nothing, until someth…
Re: PDF to Text, a challenging problem
#137Have any of you ever thought to yourself, this is new and interesting, and then vaguely remembered that you spent months or years becoming an expert at it earlier in life but entirely forgot it? And in fact large chunks of the very interesting things you've done just completely flew out of your mind long ago, to the point where you feel absolutely new at life, like you've accomplished relatively nothing, until someth…
We will never get back the collective man-decades of time that has been burned by this format. When will the madness stop?
(1) be stored in a single file
(2) Allow tables, images and anything else that can be shown on a piece paper
(3) Won't have animation, fold-out text, or anything that cannot be be shown on a piece of paper
(4) won't require Javascript or access to external sites
that means never.. We've got lucky we at least got PDF before "web designers" made (3) impossible, and marketers made (4) impossible
Re: PDF to Text, a challenging problem
#138Earlier quoted context omitted.
The better solution to a search engine extracting text from existing PDFs is to provide advice on how to author PDFs? What's the timeline for this solution to pay off
Microsoft is one of the bigger contributors to this. Like -- why does excel have a feature to export to PDF, but not a feature to do the opposite? That export functionality really feels like it was given to a summer intern who finished it in two weeks and never had to deal with it ever again.
In practice I’ve found it to be extremely unreliable, and I suspect this may be because the optional metadata that semantically defines a table as a table is missing from the errant PDF. It’ll still look like a table when rendered, but there’s nothing that defines it as such. It’s just a bunch of graphical and text elements that, when rendered, happen to look like a table.
Re: PDF to Text, a challenging problem
#139Earlier quoted context omitted.
We will never get back the collective man-decades of time that has been burned by this format. When will the madness stop?
When we get an alternative that can: (1) be stored in a single file (2) Allow tables, images and anything else that can be shown on a piece paper (3) Won't have animation, fold-out text, or anything that cannot be be shown on a piece of paper (4) won't require Javascript or access to external sites that means never.. We've got lucky we at least got PDF before "web designers" made (3) impossible, and marketers made (4…
Re: PDF to Text, a challenging problem
#140Having built some toy parsers for PDF files in the past it was a huge wtf moment for me when I realized how the format works. With that in mind, it's even more puzzling how it's used often in text-heavy cases. I always think about the invoicing use-case: digital systems should be able to easy extract data from the file while still being formatted visually for humans. It seems like the tech world would be much better…