Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I am building a new Python library to read/write PDF files

github.com

Post reply on HN
Print view View original

111–120 of 126 posts

Re: Show HN: I am building a new Python library to read/write PDF files

#111

Post by mdaniel »

svat wrote: post #52

This is really wonderful, thank you! It's great to see someone focusing on the internal structure of PDF files (the "Syntax" chapter of the spec), and doing things with a focus on browsing the internal structure etc. (I had a similar idea and did something in Rust/WASM back in May; let me see if I can dust it off and put it on GitHub. Edit: not very usable, but here FWIW: https://github.com/shreevatsa/pdf-explorer )…

  Commiting a change (from Jun 5 20:57) that I don't understand any more 

     // From real life, lightly modified. Note the "/companyName, LLC" as key!
With absolutely no slight toward the author, that matches my mental model of dealing with PDFs: `git commit -mwtf`

Re: Show HN: I am building a new Python library to read/write PDF files

#112

Post by cgdae »

cochne wrote: post #54

You’re in for it! I highly recommend checking out mupdf, it was one of the more pleasant Python libraries I dealt with for this purpose.

I think you might mean PyMuPDF (https://github.com/pymupdf/PyMuPDF), a Python library built on top of the MuPDF C library (https://mupdf.com/).

PyMuPDF and MuPDF are both available under dual open source AGPL and commercial licenses. They have been around for many years and are under continual development.

[Disclaimer, i work for Artifex, who wrote MuPDF and recently acquired PyMuPDF.]

Re: Show HN: I am building a new Python library to read/write PDF files

#113

Post by mdaniel »

userbinator wrote: post #41
jahewson wrote: post #21

Earlier quoted context omitted.

That would be XPS https://en.m.wikipedia.org/wiki/Open_XML_Paper_Specification

Anything related to XML is arguably even worse.

I know it's fun to hate on XML, but as compared to inventing a new pseudo-text-pseudo-binary format, its parsing mechanics are well understood

I'm not claiming all of PDF's woes are related to its encoding, but it's not zero, either. Start from the fact that XML documents have XML Schema allowing one to formally specify what can and cannot appear where. The PDF specification is a bunch of English which makes for shitty constraint boundaries

Re: Show HN: I am building a new Python library to read/write PDF files

#114

Post by ehutch79 »

RantyDave wrote: post #88

I once had to parse a reference manual, provided as a PDF and emit a mostly-usable CSV of its content. That shit was hard . Writing PDF is one thing but there are some psychopathic PDF's out there when you scratch below the surface. People do .... well, you'll find out.

Table elements not being in consecutive display order in any direction. Like elements 0-5 from column a, followed by elements 10-12 from column b, elements 6,8.19 from column a, then elements 19-4 in reverse order from column d, all of column c....

This is a real thing I dealt with.

Re: Show HN: I am building a new Python library to read/write PDF files

#115

Post by svat »

mdaniel wrote: post #111
svat wrote: post #52

This is really wonderful, thank you! It's great to see someone focusing on the internal structure of PDF files (the "Syntax" chapter of the spec), and doing things with a focus on browsing the internal structure etc. (I had a similar idea and did something in Rust/WASM back in May; let me see if I can dust it off and put it on GitHub. Edit: not very usable, but here FWIW: https://github.com/shreevatsa/pdf-explorer )…

Commiting a change (from Jun 5 20:57) that I don't understand any more // From real life, lightly modified. Note the "/companyName, LLC" as key! With absolutely no slight toward the author, that matches my mental model of dealing with PDFs: `git commit -mwtf`

I'm the author and I just meant I had left behind a small uncommitted diff back when I stopped working on it, and I didn't bother to read the diff before committing. I actually understand it just fine, on second look…

Overall, at least so far, I haven't encountered much "WTF" dealing with PDFs actually. The spec (especially the Adobe version: the ISO version based on it is only slightly different but feels much worse) is quite pleasant to read. There are some warts from backward compatibility with earlier poor decisions, but not too many of them. And while it's surprising what different PDF programs will produce as long as any PDF reader in existence happens to accepts it (Hyrum's law) (e.g. in this example, the dictionary key having a space in it), for my purposes it hasn't been a big deal as I'm only trying to do the first level of parsing, and when even that is problematic I can happily just declare the PDF malformed.

Re: Show HN: I am building a new Python library to read/write PDF files

#116

Post by gettalong »

As others have already written, there are many slightly invalid PDF files out there in the wild that many readers can display mostly fine and which your library should also be able to handle.

If you can, grab yourself a copy of the most recent PDF 2.0 specification since it contains much more information and is much more correct in terms of how to implement things. Also have a look at the errata at https://pdf-issues.pdfa.org/32000-2-2020/index.html.

As I'm implementing a PDF library (in Ruby), I have started to collect some situations that arise in the wild but are not spec-compliant, see https://github.com/gettalong/annotated-pdf-spec. That might help you in parsing some invalid PDFs

Re: Show HN: I am building a new Python library to read/write PDF files

#117

Post by 999900000999 »

mythrwy wrote: post #27
999900000999 wrote: post #18

Earlier quoted context omitted.

OCR is built into Adobe's PDF reader, issue is it's 15$ a month. I really want to see OCR become easier to use, but I don't know why it's such a hard problem in the first place.

There is the python library ocrmypdf https://ocrmypdf.readthedocs.io/en/latest/ that works really well. I have found the results comparable to Adobe in accuracy. I believe it uses tesseract, ghostscript and some other libraries. Speaking of ghostscript, one way to deal with problematic PDFs is to print them to file and deal with the result instead.

Any open source apps integrate this ?

I'd love to just be able to search a PDF document for a string and get a list of results.

Re: Show HN: I am building a new Python library to read/write PDF files

#118

Post by poxrud »

neilv wrote: post #4

Neat. Another use case for which you might want to think about a sample is extracting data from filled PDF forms. (That use case is why I once had to write a PDF parser.) Since you read&write, maybe also a use case of programmatically filling some form fields in an editable PDF form. Such pre-filling some of the fields for a particular Web site user in a dynamically-modified PDF form they download. But the source PDF…

Fun project story... During the first covid school shutdown my son's day care wanted parents to print a daily screening symptom checklist, take a photo of it and email it to them every morning. This was a tedious process that I automated with PyPDF2 + PDFtk + pypdftk. It's easy to generate your own PDF's but it's harder to take an existing, outdated, non-editable PDF and automatically fill it out.

Eventually I turned it into a website, added AWS API Gateway + Lambda and put the whole thing up for other daycare parents to use. Two weeks later the daycare switched to google forms and my project was not useful anymore.

Re: Show HN: I am building a new Python library to read/write PDF files

#119

Post by mdaniel »

gettalong wrote: post #116

As others have already written, there are many slightly invalid PDF files out there in the wild that many readers can display mostly fine and which your library should also be able to handle. If you can, grab yourself a copy of the most recent PDF 2.0 specification since it contains much more information and is much more correct in terms of how to implement things. Also have a look at the errata at https://pdf-issues…

Merely for your consideration, if those were actual issues on that repo, (a) it would allow adding labels to them (as in https://github.com/pdf-association/pdf-issues/issues?q=is%3A... ) (b) folks could comment, acting as a low-rent stackoverflow, and (c) it would allow anyone to contribute new ones versus the "PR against README.md" situation right now

That also more closely matches the mental model of those items: bugs against the specification, whether the official PDF Association agrees that they are or not

Re: Show HN: I am building a new Python library to read/write PDF files

#120

Post by mdaniel »

poxrud wrote: post #118
neilv wrote: post #4

Neat. Another use case for which you might want to think about a sample is extracting data from filled PDF forms. (That use case is why I once had to write a PDF parser.) Since you read&write, maybe also a use case of programmatically filling some form fields in an editable PDF form. Such pre-filling some of the fields for a particular Web site user in a dynamically-modified PDF form they download. But the source PDF…

Fun project story... During the first covid school shutdown my son's day care wanted parents to print a daily screening symptom checklist, take a photo of it and email it to them every morning. This was a tedious process that I automated with PyPDF2 + PDFtk + pypdftk. It's easy to generate your own PDF's but it's harder to take an existing, outdated, non-editable PDF and automatically fill it out. Eventually I turned…

> it's harder to take an existing, outdated, non-editable PDF and automatically fill it out.

That has been on my wishlist for several years: build a "PDF annotation" service that takes in a PDF that is not an XObject form (e.g. this random example: https://www.dentalworks.com/wp-content/uploads/2021/08/Patie... ) and replace those _____ areas with actual PDF inputs. My handwriting is terrible, and it's a waste of human capital for some poor soul to try and decipher handwriting only to (almost undoubtedly) re-type it into a computer on their end

I am sure we ended up in this situation because people just "File > Print to PDF" from Word or whatever, because knowing that PDF forms exist and then how to use Adobe(R) whatever(tm) to make a real editable PDF is "too much to ask."

I have had about 10% success with Preview.app detecting the lines and allowing me to click on them and type, but having https://notstupidpdf.example.com/www.dentalworks.com/wp-cont... would be much better for humanity

Post reply on HN