Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

So you want to parse a PDF?

eliot-jones.com

Post reply on HN
Print view View original

31–40 of 236 posts

Re: So you want to parse a PDF?

#31

Post by spankibalt »

throwaway4496 wrote: post #23
Alex3917 wrote: post #19

Earlier quoted context omitted.

> This is exactly the reason why Computer Vision approaches for parsing PDFs works so well in the real world. One of the biggest benefits of PDFs though is that they can contain invisible data. E.g. the spec allows me to embed cryptographic proof that I've worked at the companies I claim to have worked at within my resume. But a vision-based approach obviously isn't going to be able to capture that.

Cryptographic proof of job experience? Please explain more. Sounds interesting.

Encrypted (and hidden) embedded information, e. g. documents, signatures, certificates, watermarks, and the like. To (legally-binding) standards, e. g. for notary, et cetera.

Re: So you want to parse a PDF?

#32

Post by sidebute »

throwaway4496 wrote: post #20
bee_rider wrote: post #17

Earlier quoted context omitted.

Seems like a fairly reasonable decision given all the high quality implementations out there.

How is it reasonable to render the PDF, rasterize it, OCR it, use AI, instead of just using the "quality implementation" to actually get structured data out? Sounds like "I don't know programming, so I will just use AI".

> Sounds like "I don't know programming, so I will just use AI".

If you were leading Tensorlake, running on early stage VC with only 10 employees (https://pitchbook.com/profiles/company/594250-75), you'd focus all your resources on shipping products quickly, iterating over unseen customer needs that could make the business skyrocket, and making your customers so happy that they tell everyone and buy lots more licenses.

Because you're a stellar tech leader and strategist, you wouldn't waste a penny reinventing low-level plumbing that's available off-the-shelf, either cheaply or as free OSS. You'd be thinking about the inevitable opportunity costs: If I build X then I can't build Y, simply because a tiny startup doesn't have enough resources to build X and Y. You'd quickly conclude that building a homegrown, robust PDF parser would be an open-ended tar pit that precludes us from focusing on making our customers happy and growing the business.

And the rest of us would watch in awe, seeing truly great tech leadership at work, making it all look easy.

Re: So you want to parse a PDF?

#33

Post by rafram »

diptanu wrote: post #15

Disclaimer - Founder of Tensorlake, we built a Document Parsing API for developers. This is exactly the reason why Computer Vision approaches for parsing PDFs works so well in the real world. Relying on metadata in files just doesn't scale across different source of PDFs. We convert PDFs to images, run a layout understanding model on them first, and then apply specialized models like text recognition and table recogn…

This has close to zero relevance to the OP.

Re: So you want to parse a PDF?

#34

Post by leeter »

I remember having a prior boss of mine asked if the application the company I was working for made could use PDF as an input. His response was to laugh then say "No, there is no coming back from chaos." The article has only reinforced that he was right.

Re: So you want to parse a PDF?

#35

Post by koakuma-chan »

throwaway4496 wrote: post #20
bee_rider wrote: post #17

Earlier quoted context omitted.

Seems like a fairly reasonable decision given all the high quality implementations out there.

How is it reasonable to render the PDF, rasterize it, OCR it, use AI, instead of just using the "quality implementation" to actually get structured data out? Sounds like "I don't know programming, so I will just use AI".

I think it's reasonable because their models are probably trained on images, and not whatever "structured data" you may get out of a PDF.

Re: So you want to parse a PDF?

#36

Post by sidebute »

diptanu wrote: post #15

Disclaimer - Founder of Tensorlake, we built a Document Parsing API for developers. This is exactly the reason why Computer Vision approaches for parsing PDFs works so well in the real world. Relying on metadata in files just doesn't scale across different source of PDFs. We convert PDFs to images, run a layout understanding model on them first, and then apply specialized models like text recognition and table recogn…

While we have a PDF internals expert here, I'm itching to ask: Why is mupdf-gl so much faster than everything else? (on vanilla desktop linux)

Its search speed on big pdfs is dramatically faster than everything else I've tried and I've often wondered why the others can't be as fast as mupdf-gl.

Thanks for any insights!

Re: So you want to parse a PDF?

#37

Post by reactordev »

throwaway4496 wrote: post #20
bee_rider wrote: post #17

Earlier quoted context omitted.

Seems like a fairly reasonable decision given all the high quality implementations out there.

How is it reasonable to render the PDF, rasterize it, OCR it, use AI, instead of just using the "quality implementation" to actually get structured data out? Sounds like "I don't know programming, so I will just use AI".

As someone who had to parse form data from a pdf, where the pdf author named the inputs TextField1 TextField2 TextFueld3 etc.

Misspellings, default names, a mixture, home brew naming schemes, meticulous schemes, I’ve seen it all. It’s definitely easier to just rasterize it and OCR it.

Re: So you want to parse a PDF?

#39

Post by achillesheels »

diptanu wrote: post #15

Disclaimer - Founder of Tensorlake, we built a Document Parsing API for developers. This is exactly the reason why Computer Vision approaches for parsing PDFs works so well in the real world. Relying on metadata in files just doesn't scale across different source of PDFs. We convert PDFs to images, run a layout understanding model on them first, and then apply specialized models like text recognition and table recogn…

Thanks for the pointer!

Re: So you want to parse a PDF?

#40

Post by bsder »

throwaway4496 wrote: post #20
bee_rider wrote: post #17

Earlier quoted context omitted.

Seems like a fairly reasonable decision given all the high quality implementations out there.

How is it reasonable to render the PDF, rasterize it, OCR it, use AI, instead of just using the "quality implementation" to actually get structured data out? Sounds like "I don't know programming, so I will just use AI".

> How is it reasonable to render the PDF, rasterize it, OCR it, use AI, instead of just using the "quality implementation" to actually get structured data out?

Because the underlying "structured data" is never checked while the visual output is checked by dozens of people.

"Truth" is the stuff that the meatbags call "truth" as seen by their squishy ocular balls--what the computer sees doesn't matter.

Post reply on HN