Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

So you want to modify the text of a PDF by hand (2020)

gist.github.com

Post reply on HN
Print view View original

21–30 of 103 posts

Re: So you want to modify the text of a PDF by hand (2020)

#21

Post by jl6 »

This seems to be missing an important step in the use of qpdf’s --qdf mode: after you’ve finished editing, you need to run the file through the fix-pdf utility to recalculate all the object offsets and rebuild the cross-reference table that lives at the end of the file (unless you only change bytes in-place rather than adding or removing bytes).

My top 3 fun PDF facts:

1) Although PDF documents are typically 8-bit binary files, you can make one that is valid UTF-8 “plain text”, even including images, through the use of the ASCII85 filter.[0]

2) PDF allows an incredible variety of freaky features (3D objects, JavaScript, movies in an embedded flash object, invisible annotations…). PDF/A is a much saner, safer subset.

3) The PDF spec allows you to write widgets (e.g. form controls) using “rich text”, which is a subset of XHTML and CSS - but this feature is very sparsely supported outside the official Adobe Reader.

[0] For example: https://lab6.com/2

Re: So you want to modify the text of a PDF by hand (2020)

#23

Post by aidos »

blincoln wrote: post #11

The PDF specification is wild. My current favourite trivia is that it supports all of Photoshop's layer blend modes for rendering overlapping elements.[1] My second-favourite is that it supports appended content that modifies earlier content, so one should always look for forensic evidence in all distinct versions represented in a given file.[2] It's also a fun example of the futility of DRM. The spec includes passwo…

To be fair, if you wanted to stop copying of text it would be easiest just to drop the ToUnicode mapping against the fonts and then it’s a manual process for people to recreate them.

Re: So you want to modify the text of a PDF by hand (2020)

#24

Post by bena »

ks2048 wrote: post #8

This seems to be missing an important point: at the end of PDF is a table ("cross-reference" table) that stores the BYTE-OFFSET to different objects in the file. If you modify things within the file, typically these offsets will change and the file will be corrupt. It looks like in this article, maybe they were only interested in changing one number to another, so none of the positions change. But, generally, adding/…

Ah. So it's a lot like editing compiled binaries.

You can modify binaries all you want as long as you preserve the length of everything.

Some piece of software we had authenticated against a server, but everything was done on the client. The client executed SQL against the server directly, etc. Basically, the server checked to see if this client would put you over the number of licenses you purchased and that's it.

I had run it against a disassembler, found the part where it performed the check, and was able to change it to a straight JMP and then pad the rest of the space with NOPs.

Re: So you want to modify the text of a PDF by hand (2020)

#25

Post by aardvark179 »

blincoln wrote: post #11

The PDF specification is wild. My current favourite trivia is that it supports all of Photoshop's layer blend modes for rendering overlapping elements.[1] My second-favourite is that it supports appended content that modifies earlier content, so one should always look for forensic evidence in all distinct versions represented in a given file.[2] It's also a fun example of the futility of DRM. The spec includes passwo…

Aren’t the blend modes supported just the Porter-Duff compositing modes? You might think that’s overkill, but it’s a really good mapping of what other rendering pipelines offer and it can really help reduce the work to produce a PDF.

Re: So you want to modify the text of a PDF by hand (2020)

#26

Post by gpvos »

ks2048 wrote: post #8

This seems to be missing an important point: at the end of PDF is a table ("cross-reference" table) that stores the BYTE-OFFSET to different objects in the file. If you modify things within the file, typically these offsets will change and the file will be corrupt. It looks like in this article, maybe they were only interested in changing one number to another, so none of the positions change. But, generally, adding/…

That's why they decode it with qpdf and re-encode it again afterwards, so qpdf takes care of that. qpdf reconstructs the original PDF structure, and I think it even tries to keep the object numbers the same, but the offsets are recalculated completely.

Re: So you want to modify the text of a PDF by hand (2020)

#27

Post by gpvos »

jl6 wrote: post #21

This seems to be missing an important step in the use of qpdf’s --qdf mode: after you’ve finished editing, you need to run the file through the fix-pdf utility to recalculate all the object offsets and rebuild the cross-reference table that lives at the end of the file (unless you only change bytes in-place rather than adding or removing bytes). My top 3 fun PDF facts: 1) Although PDF documents are typically 8-bit bi…

After you've finished editing, just run it through qpdf without parameters, as explained in the beginning of the article, and it will recompress the data and recreate the xref table. No need for yet another tool.

Re: So you want to modify the text of a PDF by hand (2020)

#28

Post by gpvos »

aleden wrote: post #9

I'm surprised no one has mentioned qpdf. https://qpdf.readthedocs.io/en/stable/overview.html It turns a PDF (typically everything in it is compressed binary blobs) into a mixed binary/ASCII file (which itself is a PDF) that can be edited with vim.

I'm not sure what you were reading, but the fine article is centred around using qpdf.

Re: So you want to modify the text of a PDF by hand (2020)

#29

Post by Someone »

userbinator wrote: post #14
ks2048 wrote: post #8

This seems to be missing an important point: at the end of PDF is a table ("cross-reference" table) that stores the BYTE-OFFSET to different objects in the file. If you modify things within the file, typically these offsets will change and the file will be corrupt. It looks like in this article, maybe they were only interested in changing one number to another, so none of the positions change. But, generally, adding/…

That's the weirdest part of the PDF spec IMHO. It's a mix of both binary and text, with text-specified byte offsets. It would be very interesting to read about why the format became like that, if its authors would ever talk about it. My guess is that it was meant to be completely textual at first (but then requiring the xref table to have fixed-length entries is odd), and then they decided binary would be more effici…

> My guess is that it was meant to be completely textual at first

It indeed started life as “not Turing complete postscript with an index” (those makes it easy to render just the third page of a PDF file, something that’s impossible in postscript without rendering the first and second pages first). Like postscript, that was a pure text format.

One nice feature is that you can append a few pieces and a new index to an existing PDF file and get a new valid PDF file (which would still contain its old index as a piece of “junk DNA”)

I think compression was added because users complained about file sizes. Ascii85 (https://en.m.wikipedia.org/wiki/Ascii85) grows binary data by 25%.

> but then requiring the xref table to have fixed-length entries is odd

My guess is that made it easier to hack together a tool to convert PDF to postscript.

Re: So you want to modify the text of a PDF by hand (2020)

#30

Post by miki123211 »

aidos wrote: post #23
blincoln wrote: post #11

The PDF specification is wild. My current favourite trivia is that it supports all of Photoshop's layer blend modes for rendering overlapping elements.[1] My second-favourite is that it supports appended content that modifies earlier content, so one should always look for forensic evidence in all distinct versions represented in a given file.[2] It's also a fun example of the futility of DRM. The spec includes passwo…

To be fair, if you wanted to stop copying of text it would be easiest just to drop the ToUnicode mapping against the fonts and then it’s a manual process for people to recreate them.

That also breaks search (and more importantly screen reader accessibility), and if you're professionally required to specifically produce PDFs with these security features enabled, you're pretty likely to be working in a context where that would be illegal.
Post reply on HN