Live data from Hacker News

What's so hard about PDF text extraction?

filingdb.com

21–30 of 350 posts

Re: What's so hard about PDF text extraction?

#21
post #14

On a personal project, I had a good experience extracting PDF text using Tabula[1]. You specify the bounding boxes where desired data is, and it spits out the content it finds. It still hits the issues mentioned in this article (surprise spaces appearing in middle of words, etc) [1] https://tabula.technology/

Does it also extract PDF annotations?

Re: What's so hard about PDF text extraction?

#22

So all through this I’m thinking “just OCR it and be done”, and we get to: > Why not OCR all the time? > Running OCR on a PDF scan usually takes at least an order of magnitude longer than extracting the text directly from the PDF. ... so? Google can OCR video and translate it in something that feels like real-time; what PDF processing are they doing that is so performance bound? > Difficulties with non-standard chara…

> Google can

1) Do you have trillion or so dollars at your beck and call? If not, you're not Google.

> Finding an algorithm that approximates how a human...

2) ...is generally nigh impossible even for someone with Google's resources (e.g. Waymo, although when it comes to reading, it's somewhat usable). Also, look at 1)

Unless by approximate you mean toddler level. In that case:

3) The approximation is probably useless

Re: What's so hard about PDF text extraction?

#23
post #8

The relatively small company I work for makes me fill out some forms by hand, because they receive them from vendors as a PDF. So I print it out, sign it, and return it to my company by hand. If someone could make a service that lets you upload a PDF that contains a form, and then let users fill out that form and e-sign it and collect the results, and then print them out all at once, it would be great. It's not a bil…

There are quite a few services that should be able to solve this problem (turning a PDF into a web form and collecting signatures.) Here's a few of the services I'm aware of:

* https://www.hellosign.com/products/helloworks

* https://www.useanvil.com

* https://www.pandadoc.com

* https://www.pdffiller.com

* https://www.platoforms.com

* JotForm (https://www.jotform.com/help/433-How-to-Add-an-E-Signature-t...)

* https://www.webmerge.me

(I know about all these because I'm working on a PDF generation service for developers called DocSpring [1]. I'm also working on e-signature support [2], but that's still under development, and still won't be a perfect fit for your use-case.)

[1] https://docspring.com

[2] https://docspring.com/docs/data_requests.html

Re: What's so hard about PDF text extraction?

#24
I worked on PDF generating software for years. It's a horrible format that should never have been approved as an ISO standard.

When in doubt, use plain text. It's a million times better in every way that counts.

I wish my bank statements and such could be downloaded as plain text files, instead of massive PDF files that embed another copy of a bunch of typefaces in each file.

Re: What's so hard about PDF text extraction?

#25
I’m a contractor. One of my gigs involved writing parsers for 20-something different kinds of pdf bank statements. It’s a dark art. Once you’ve done it 20 times it becomes a lot easier. Now we simply POST a pdf to my service and it gets parsed and the data it contains gets chucked into a database. You can go extremely far with naive parsers. That is, regex combined with positionally-aware fixed-length formatting rules. I’m available for hire re. structured extraction from PDFs. I’ve also got a few OCR tricks up my sleeve (eg for when OCR thinks 0 and 6 are the same)

Re: What's so hard about PDF text extraction?

#27

So all through this I’m thinking “just OCR it and be done”, and we get to: > Why not OCR all the time? > Running OCR on a PDF scan usually takes at least an order of magnitude longer than extracting the text directly from the PDF. ... so? Google can OCR video and translate it in something that feels like real-time; what PDF processing are they doing that is so performance bound? > Difficulties with non-standard chara…

> Finding an algorithm that approximates how a human approaches a page layout doesn’t feel like it would be all that hard.

"In CS, it can be hard to explain the difference between the easy and the virtually impossible."

https://xkcd.com/1425/

Re: What's so hard about PDF text extraction?

#28
post #8

The relatively small company I work for makes me fill out some forms by hand, because they receive them from vendors as a PDF. So I print it out, sign it, and return it to my company by hand. If someone could make a service that lets you upload a PDF that contains a form, and then let users fill out that form and e-sign it and collect the results, and then print them out all at once, it would be great. It's not a bil…

hellosign.com does exactly this

Re: What's so hard about PDF text extraction?

#29
> By looking at the content, understanding what it is talking about and knowing that vegetables are washed before chopping, we can determine that A C B D is the correct order. Determining this algorithmically is a difficult problem.

Sorry, this is a bit off-topic regarding PDF extraction, but it distracted me greatly while reading...

I'm pretty sure the intention was A B C D (cut then wash). Not sure why the author would not use alphabet order for the recipe...

[edit] Sorry, I made it read to a colleague and he mentioned the A B C D annotations were probably not in the original document. This was not clear at all for me while reading, and if they are not included it's indeed hard to find the correct paragraph order.

Re: What's so hard about PDF text extraction?

#30
post #19
post #8

The relatively small company I work for makes me fill out some forms by hand, because they receive them from vendors as a PDF. So I print it out, sign it, and return it to my company by hand. If someone could make a service that lets you upload a PDF that contains a form, and then let users fill out that form and e-sign it and collect the results, and then print them out all at once, it would be great. It's not a bil…

I use Xournal - https://sourceforge.net/projects/xournal/ It lets me type in to forms - or draw text over them if necessary. Then I paste in a scan of my signature. Then save as a PDF an email across. I've been doing this for years. Job applications, mortgages, medical questionnaires. No one has every queried it. If you're hand delivering a printed PDF, it's just going to be copy-typed by a human into a computer. No…

macOS's preview will let you do all of this also.
Post reply on HN