Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New information extracted from Snowden PDFs through metadata version analysis

libroot.org

Post reply on HN
Print view View original

101–110 of 139 posts

Re: New information extracted from Snowden PDFs through metadata version analysis

#101

Post by cubefox »

PhilipRoman wrote: post #95
cubefox wrote: post #72

Earlier quoted context omitted.

Aren't there file systems that support data structures which allow editing just part of the data, like linked lists?

Yeah there are, Linux supports parameters FALLOC_FL_INSERT_RANGE and FALLOC_FL_COLLAPSE_RANGE for fallocate(2). Like most fancy filesystem features, they are not used by the vast majority of software because it has to run on any filesystem so you'd always need to maintain two implementations (and extensive test cases).

Interesting that after decades of file system history, this is still considered a "fancy feature", considering that editing files is a pretty basic operation for a file system. Though I assume there are reasons why this hasn't become standard long ago.

Re: New information extracted from Snowden PDFs through metadata version analysis

#102

Post by Ylpertnodi »

rendx wrote: post #96
tolerance wrote: post #75

[flagged]

Are you asking how much was done with pen and paper, and how much of it was done on a computer, i.e. machine assisted? Where do you draw the line? How is "hands-on" in contrast to anything? Is it only "hands-on" when you don't use any tool to assist you? I suspect you're inquiring about the use of LLMs, and about that I wonder: Why does it matter? Why are you asking?

[flagged]

Re: New information extracted from Snowden PDFs through metadata version analysis

#103

Post by rendx »

Ylpertnodi wrote: post #102
rendx wrote: post #96

Earlier quoted context omitted.

Are you asking how much was done with pen and paper, and how much of it was done on a computer, i.e. machine assisted? Where do you draw the line? How is "hands-on" in contrast to anything? Is it only "hands-on" when you don't use any tool to assist you? I suspect you're inquiring about the use of LLMs, and about that I wonder: Why does it matter? Why are you asking?

[flagged]

Are you confusing me with the authors, or why would you think I could? And I'm asking 'tolerance' to clarify their question, which means I wouldn't be able to answer it even if I had the knowledge they were after, since I don't understand what they're asking.

Re: New information extracted from Snowden PDFs through metadata version analysis

#104

Post by formerly_proven »

PhilipRoman wrote: post #95
cubefox wrote: post #72

Earlier quoted context omitted.

Aren't there file systems that support data structures which allow editing just part of the data, like linked lists?

Yeah there are, Linux supports parameters FALLOC_FL_INSERT_RANGE and FALLOC_FL_COLLAPSE_RANGE for fallocate(2). Like most fancy filesystem features, they are not used by the vast majority of software because it has to run on any filesystem so you'd always need to maintain two implementations (and extensive test cases).

What this does on typical extent-based file systems is split the extent of the file at the given location (which means these operations can only be done with cluster granularity) and then insert a third extent. i.e. calling INSERT_RANGE once will give you a file with at least three extents (fragments). This, plus the mkfs-options-dependent alignment requirements, makes it really quite uninteresting for broad use in a similar fashion as O_DIRECT is uninteresting.

Re: New information extracted from Snowden PDFs through metadata version analysis

#105

Post by octoberfranklin »

culi wrote: post #88
cookiengineer wrote: post #37

Earlier quoted context omitted.

> The "print and scan physical papers back to a PDF of images" technique for final release is looking better and better from an information protection perspective. Note that all (edit: color-/ink-) printers have "invisible to the human eye" yellow dotcodes, which contain their serial number, and in some cases even the public IP address when they've already connected to the internet (looking at you, HP and Canon). So…

That's why I'm (still) waiting on this https://www.crowdsupply.com/open-tools/open-printer It's mindboggling how much open-source 3d printing stuff is out there (and I'm grateful for it) but this is completely lacking in the 2d printing world

Sorry, you have been blocked You are unable to access crowdsupply.com Why have I been blocked? This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

What can I do to resolve this? You can email the site owner to let them know you were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

Cloudflare Ray ID: 9bbed59d7bcd9dfc • Performance & security by Cloudflare

Re: New information extracted from Snowden PDFs through metadata version analysis

#106

Post by maweki »

layer8 wrote: post #71
cubefox wrote: post #65

Earlier quoted context omitted.

I'm pretty sure you can change various file formats without rewriting the entire file and without using "incremental updates".

You can’t insert data into the middle of a file (or remove portions from the middle of a file) without either rewriting it completely, or at least rewriting everything after the insertion point; the latter requires holding everything after the insertion point in memory (or writing it out to another file first, then reading it in and writing it out again). PDF is designed to not require holding the complete file in me…

While tedious, you can do the rewrite block-wise from the insertion point and only store a an additional block's worth of the rest (or twice as much as you inserted)

ABCDE, to insert 1 after C: store D, overwrite D with 1, store E, overwrite E with D, write E.

Re: New information extracted from Snowden PDFs through metadata version analysis

#107

Post by [deleted] »

cubefox wrote: post #65
layer8 wrote: post #61

Earlier quoted context omitted.

I don’t think this was particularly modeled on MS Word. The incremental update feature was introduced with PDF 1.2 in 1996. It allows to quickly save changes without having to rewrite the whole file, for example when annotating a PDF. Incremental updates are also essential for PDF signatures, since when you add a subsequent signature to a PDF, you couldn’t rewrite the file without breaking previous signatures. Hence…

I'm pretty sure you can change various file formats without rewriting the entire file and without using "incremental updates".

[deleted]

Re: New information extracted from Snowden PDFs through metadata version analysis

#108

Post by layer8 »

cubefox wrote: post #101
PhilipRoman wrote: post #95

Earlier quoted context omitted.

Yeah there are, Linux supports parameters FALLOC_FL_INSERT_RANGE and FALLOC_FL_COLLAPSE_RANGE for fallocate(2). Like most fancy filesystem features, they are not used by the vast majority of software because it has to run on any filesystem so you'd always need to maintain two implementations (and extensive test cases).

Interesting that after decades of file system history, this is still considered a "fancy feature", considering that editing files is a pretty basic operation for a file system. Though I assume there are reasons why this hasn't become standard long ago.

File systems aren’t databases; they manage flat files, not structured data. You also can’t just insert/remove random amounts of bytes in RAM. The considerations here are actually quite similar, like fragmentation. If you make a hundred small edits to a file, you might end up with the file taking up ten times as much space due to fragmentation, and then you’d need the file system to do some sort of defragmentation pass to rewrite the file more contiguously again.

In addition, it’s generally nontrivial for a program to map changes to an in-memory object structure back to surgical edits of a flat file. It’s much easier to always just serialize the whole thing, or if the file format allows it, appending the serialized changes to the file.

Re: New information extracted from Snowden PDFs through metadata version analysis

#109

Post by dredmorbius »

IshKebab wrote: post #84
dredmorbius wrote: post #79

Earlier quoted context omitted.

Far more straightforward to print a stack, then feed that stack through the copier/scanner.

You don't need to actually print and scan. Just convert to a raster format like PNG.

Given issues with fully-electronic conversion, passing through a paper phase tends to guard against foul-ups. It's tangible and demonstrable. People ... take short-cuts, which is why we're having this discussion.

Re: New information extracted from Snowden PDFs through metadata version analysis

#110

Post by theturtletalks »

layer8 wrote: post #50

These PDFs apparently used the “incremental update” feature of PDF, where edits to the document are merely appended to the original file. It’s easy to extract the earlier versions, for example with a plain text editor. Just search for lines starting with “%%EOF”, and truncate the file after that line. Voila, the resulting file is the respective earlier PDF version. (One exception is the first %%EOF in a so-called lin…

New OSINT skill unlocked
Post reply on HN