Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What's so hard about PDF text extraction?

filingdb.com

Post reply on HN
Print view View original

31–40 of 350 posts

Re: What's so hard about PDF text extraction?

#31

Post by maest »

Santosh83 wrote: post #2

Another site that breaks the browser's back navigation. Why do so many sites do this? Do they imagine they retain user attention for longer if they break navigation? It's pretty trivial to long-press the back button or just close the tab and not come back again to your site...

Hi, author here.

We've taken no intentional action to change the way the back button works - in fact, I too hate it when websites do that.

Can you PM me with some details about what you're seeing? I'm having issues reporducing it with my particular setup.

Re: What's so hard about PDF text extraction?

#32

Post by pintxo »

jatsign wrote: post #8

The relatively small company I work for makes me fill out some forms by hand, because they receive them from vendors as a PDF. So I print it out, sign it, and return it to my company by hand. If someone could make a service that lets you upload a PDF that contains a form, and then let users fill out that form and e-sign it and collect the results, and then print them out all at once, it would be great. It's not a bil…

iPad OS should also be able to do this. Especially if you have the pen.

Re: What's so hard about PDF text extraction?

#33

Post by hippish »

amelius wrote: post #4
Santosh83 wrote: post #2

Another site that breaks the browser's back navigation. Why do so many sites do this? Do they imagine they retain user attention for longer if they break navigation? It's pretty trivial to long-press the back button or just close the tab and not come back again to your site...

This was done to impede automatic HTML text extraction :)

I'm curious, how would fiddling with navigation impede text extraction? The site renders perfectly fine without javascript, and the markup looks straight forward enough

Re: What's so hard about PDF text extraction?

#34

Post by slig »

daniel-levin wrote: post #25

I’m a contractor. One of my gigs involved writing parsers for 20-something different kinds of pdf bank statements. It’s a dark art. Once you’ve done it 20 times it becomes a lot easier. Now we simply POST a pdf to my service and it gets parsed and the data it contains gets chucked into a database. You can go extremely far with naive parsers. That is, regex combined with positionally-aware fixed-length formatting rule…

Maybe there's a SasS opportunity for you to explore.

Re: What's so hard about PDF text extraction?

#35

Post by upofadown »

hwc wrote: post #24

I worked on PDF generating software for years. It's a horrible format that should never have been approved as an ISO standard. When in doubt, use plain text. It's a million times better in every way that counts. I wish my bank statements and such could be downloaded as plain text files, instead of massive PDF files that embed another copy of a bunch of typefaces in each file.

I think that root problem here is that most people still have trouble separating the data from the presentation. We have to understand that in the end, substance always beats form.

Re: What's so hard about PDF text extraction?

#36

Post by throwaway3157 »

daniel-levin wrote: post #25

I’m a contractor. One of my gigs involved writing parsers for 20-something different kinds of pdf bank statements. It’s a dark art. Once you’ve done it 20 times it becomes a lot easier. Now we simply POST a pdf to my service and it gets parsed and the data it contains gets chucked into a database. You can go extremely far with naive parsers. That is, regex combined with positionally-aware fixed-length formatting rule…

Do you have a blog? I'd enjoy reading some of your tricks.

Also, how do you manage things when one of those banks decides to change the layout/format?

Re: What's so hard about PDF text extraction?

#37

Post by Wiretrip »

jrandm wrote: post #27
peteretep wrote: post #20

So all through this I’m thinking “just OCR it and be done”, and we get to: > Why not OCR all the time? > Running OCR on a PDF scan usually takes at least an order of magnitude longer than extracting the text directly from the PDF. ... so? Google can OCR video and translate it in something that feels like real-time; what PDF processing are they doing that is so performance bound? > Difficulties with non-standard chara…

> Finding an algorithm that approximates how a human approaches a page layout doesn’t feel like it would be all that hard. "In CS, it can be hard to explain the difference between the easy and the virtually impossible." https://xkcd.com/1425/

Totally agree. I worked on a project that had to try and extract tables from PDFs. It is much harder that it would first appear.

Re: What's so hard about PDF text extraction?

#39

Post by oddthink »

hwc wrote: post #24

I worked on PDF generating software for years. It's a horrible format that should never have been approved as an ISO standard. When in doubt, use plain text. It's a million times better in every way that counts. I wish my bank statements and such could be downloaded as plain text files, instead of massive PDF files that embed another copy of a bunch of typefaces in each file.

Ugh, this. I still fail to understand how a device from 2019, even a phone, could show any rendering delay when scrolling to page 200 of a 400 page static document. I thought PDF was less programmable than PostScript, but there's still got to be some kind of non-local semantics in there.

Re: What's so hard about PDF text extraction?

#40

Post by shawnz »

Savageman wrote: post #29

> By looking at the content, understanding what it is talking about and knowing that vegetables are washed before chopping, we can determine that A C B D is the correct order. Determining this algorithmically is a difficult problem. Sorry, this is a bit off-topic regarding PDF extraction, but it distracted me greatly while reading... I'm pretty sure the intention was A B C D (cut then wash). Not sure why the author w…

Even if the ABCD was in the original document, how would the computer figure out it's supposed to indicate the order?

And of course, even if the letters were there in the original document, it would be clear to a human that they're incorrect because it doesn't make sense to wash vegetables after cutting.

Post reply on HN