Live data from Hacker News

New information extracted from Snowden PDFs through metadata version analysis

libroot.org

111–120 of 139 posts

Re: New information extracted from Snowden PDFs through metadata version analysis

#111
post #108

Earlier quoted context omitted.

Interesting that after decades of file system history, this is still considered a "fancy feature", considering that editing files is a pretty basic operation for a file system. Though I assume there are reasons why this hasn't become standard long ago.

File systems aren’t databases; they manage flat files, not structured data. You also can’t just insert/remove random amounts of bytes in RAM. The considerations here are actually quite similar, like fragmentation. If you make a hundred small edits to a file, you might end up with the file taking up ten times as much space due to fragmentation, and then you’d need the file system to do some sort of defragmentation pas…

Indeed, also userspace-level atomicity is important, so you probably want to save a backup in case power goes out at an unfortunate moment. And since you already need to have a backup, might as well go for a full rewrite + rename combo.

Re: New information extracted from Snowden PDFs through metadata version analysis

#112
post #55

Earlier quoted context omitted.

I suppose I'd just save the pdf to tiff/png then remake back into a pdf from there to avoid printing and scanning? if really paranoid, I suppose one could run a filter on the image files to make them a bit fuzzy/noisy

I think "Print to PDF" would be easiest

I'd worry print to pdf might be ineffective. I think rasterizing is the way to go.

Re: New information extracted from Snowden PDFs through metadata version analysis

#114
post #49

Earlier quoted context omitted.

Why not just make screenshoot of every PDF page?

It could still be identifiable, for example if the document has been prepared such that the intended recipient's identity is encoded into subtle modulation of the widths of spaces.

In PDF file format?

Re: New information extracted from Snowden PDFs through metadata version analysis

#116

Earlier quoted context omitted.

> The "print and scan physical papers back to a PDF of images" technique for final release is looking better and better from an information protection perspective. Note that all (edit: color-/ink-) printers have "invisible to the human eye" yellow dotcodes, which contain their serial number, and in some cases even the public IP address when they've already connected to the internet (looking at you, HP and Canon). So…

Thanks for the links but can you share evidence for the "public IP address" claim? Each time I've read this concept (intriguing! possible!), I search for evidence and I can't find any. The MIC and yellow dots have been studied and decoded by many and all I've ever seen, including at your links, are essentially date + time + serial#. Don't get me wrong ... stamping our documents with a fingerprint back to our printers…

[deleted]

Re: New information extracted from Snowden PDFs through metadata version analysis

#117
post #96

[flagged]

Are you asking how much was done with pen and paper, and how much of it was done on a computer, i.e. machine assisted? Where do you draw the line? How is "hands-on" in contrast to anything? Is it only "hands-on" when you don't use any tool to assist you? I suspect you're inquiring about the use of LLMs, and about that I wonder: Why does it matter? Why are you asking?

First thanks for taking my question seriously and not as just a rib and asking a lot of questions in return that I want to consider myself.

By "hands-on" I'm asking whether the provided insight is the product of human intellection. Experienced, capable and qualified. Or at least an earnest attempt at thinking about something and explaining the discoveries in the ways that thinking was done before ChatGPT. For some reason I find myself using phrases involving the hands (etc. hands-on, handmade, hand-spun) as a metaphor for work done without the use of LLMs.

I emphasize insight because I feel like the series of work on the Snowden documents by libroot is wanting in that. I expressed as much the last time their writing hit the front page: https://news.ycombinator.com/item?id=46236672>.

These are summaries. I don't think that it yields information that can't otherwise be pointed out and made mention of by others; presumably known and reputable. With as high-profile of an event that this is I'd expect someone covering it almost 16 years later to tell us beyond what when judged on the merit of its import amounts to a motivated section of the ‘Snowden disclosures’ Wikipedia entry.

The discussion that this series invites typically is centered around people's thoughts about the story of the Snowden documents in general, and in this case exchanges about technical aspects like how PDF documents work and can be manipulated in general. The one comment that I feel addresses the actual tension embedded in the article—"Who edited the documents?"—leads to accusations that the documents were tampered with by the media: https://news.ycombinator.com/item?id=46566372>. I don't think that that's an implausible claim but I find issue with it being made with such confidence by the anonymous source behind the investigations (I'm withholding ironically putting "investigations" in...nevermind).

If the author actually provided something that advanced to the reader why this information is significant, what to do with or think about it and how they came about discovering the answers to the aforementioned 'why' and ‘what’ and additionally why they’re word ought to matter to us at all, I'd be less inclined to speculate that this is just someone vibe sleuthing their way through documents that on the surface are only significant to the public as the claim "the government is spying on you" is.

This particular post uncovers some nice information. It's a great find. I'm in no position to investigate whether it was already known. But what are we supposed to learn from it aside from "one of the documents were changed before it was made public". What's significant about the redaction? Is Ryan Gallagher responsible? Or does he know who is. Is he at all obliged to explain this to a presumably anonymous inquirer? Or is it now the duty of the public to expect an explanation as affected by said anonymous inquirer?

Remember when believing that the government was rife with pedophiles automatically associated you with horn-helmet-wearing insurrectionists?

Re: New information extracted from Snowden PDFs through metadata version analysis

#118

The "print and scan physical papers back to a PDF of images" technique for final release is looking better and better from an information protection perspective.

> The "print and scan physical papers back to a PDF of images" technique for final release is looking better and better from an information protection perspective. Note that all (edit: color-/ink-) printers have "invisible to the human eye" yellow dotcodes, which contain their serial number, and in some cases even the public IP address when they've already connected to the internet (looking at you, HP and Canon). So…

Could this be circumvented by randomly (or not-so-randomly) adding single-pixel yellow dots to the data sent to the printer?

Re: New information extracted from Snowden PDFs through metadata version analysis

#119
post #50

These PDFs apparently used the “incremental update” feature of PDF, where edits to the document are merely appended to the original file. It’s easy to extract the earlier versions, for example with a plain text editor. Just search for lines starting with “%%EOF”, and truncate the file after that line. Voila, the resulting file is the respective earlier PDF version. (One exception is the first %%EOF in a so-called lin…

New OSINT skill unlocked

I see an interesting parallel to how people think about captured encrypted data, and how long that encryption needs to be effective for until technology catches up and can decrypt (by which point, hopefully the decrypted data is worthless). If all of these documents are stored in durable archives, future methodologies may arrive to extract value or intelligence not originally available at the time of capture and disclosure.

Re: New information extracted from Snowden PDFs through metadata version analysis

#120
post #65
post #61

Earlier quoted context omitted.

I don’t think this was particularly modeled on MS Word. The incremental update feature was introduced with PDF 1.2 in 1996. It allows to quickly save changes without having to rewrite the whole file, for example when annotating a PDF. Incremental updates are also essential for PDF signatures, since when you add a subsequent signature to a PDF, you couldn’t rewrite the file without breaking previous signatures. Hence…

I'm pretty sure you can change various file formats without rewriting the entire file and without using "incremental updates".

This was 1996. A typical computer had tens of megabytes of memory with throughput a fraction of what we have today. Appending an element instead of reading, parsing, inserting and validating the entire document is a better solution in so many ways. That people doing redactions don't understand the technology is a separate problem. The context matters.
Post reply on HN