Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Preventing ZIP parser confusion attacks on Python package installers

blog.pypi.org

Post reply on HN
Print view View original

11–18 of 18 posts

Re: Preventing ZIP parser confusion attacks on Python package installers

#11

Post by zahlman »

calebbrown wrote: post #10
captn3m0 wrote: post #3

Now I am curious at whether these ZIP confusion attacks are mitigated at other registries that use ZIPs? Are there any such?

Apart from Python Wheels, the other popular ecosystems using zip files are Java jar files, and NuGet. Of these Java is the most interesting as there a few JDKs commonly in use. But I’m also interested in various security scanners that are built in other languages that can be fooled.

Does NPM not use zip files?

(Search results for `npm package format` are entirely not useful for figuring out what an NPM package actually consists of, beyond containing a `package.json` file. `pypi package format` results look wildly different; the first result I get is https://packaging.python.org/en/latest/discussions/package-f... which is quite comprehensive about the exact information I want — disregarding for a moment the fact that I already know this stuff ;) The NPM search results, for me, start with a Geeks4Geeks tutorial on creating a package. Is there even anything analogous to the Python Packaging Authority — misunderstood and not-actually-authoritative as it is — for NPM?)

Re: Preventing ZIP parser confusion attacks on Python package installers

#12

Post by pornel »

zahlman wrote: post #11
calebbrown wrote: post #10

Earlier quoted context omitted.

Apart from Python Wheels, the other popular ecosystems using zip files are Java jar files, and NuGet. Of these Java is the most interesting as there a few JDKs commonly in use. But I’m also interested in various security scanners that are built in other languages that can be fooled.

Does NPM not use zip files? (Search results for `npm package format` are entirely not useful for figuring out what an NPM package actually consists of, beyond containing a `package.json` file. `pypi package format` results look wildly different; the first result I get is https://packaging.python.org/en/latest/discussions/package-f... which is quite comprehensive about the exact information I want — disregarding for a…

npm and Cargo use gzipped tarballs.

Tar is an awful format that has multiple ways of specifying file names and file sizes, so there could be some shenanigans happening.

It's also possible to make archives have different content based on case-sensitivity of the file system.

Re: Preventing ZIP parser confusion attacks on Python package installers

#13

Post by zahlman »

pornel wrote: post #12
zahlman wrote: post #11

Earlier quoted context omitted.

Does NPM not use zip files? (Search results for `npm package format` are entirely not useful for figuring out what an NPM package actually consists of, beyond containing a `package.json` file. `pypi package format` results look wildly different; the first result I get is https://packaging.python.org/en/latest/discussions/package-f... which is quite comprehensive about the exact information I want — disregarding for a…

npm and Cargo use gzipped tarballs. Tar is an awful format that has multiple ways of specifying file names and file sizes, so there could be some shenanigans happening. It's also possible to make archives have different content based on case-sensitivity of the file system.

Ah. Python source distributions are the same, so there may be additional considerations there. Though in general it doesn't seem like there's much concern in the Python ecosystem about that, considering that building them will run arbitrary code anyway....

Re: Preventing ZIP parser confusion attacks on Python package installers

#14

Post by burnt-resistor »

Related to multiple .zip formats: I've found macOS Archive Utility sometimes refuses to extract early pkzip .zips created on MS-DOS, but yet Info-ZIP handles them just fine.

And, the macOS Archive Utility will complain that a proper .tar.bz2 is "corrupt" created using bzip2.

In general, be liberal in input and be conservative in output. Sometimes, this means using less features or certain older formats so that all/most things work without issues.

Re: Preventing ZIP parser confusion attacks on Python package installers

#15

Post by quietbritishjim »

burnt-resistor wrote: post #14

Related to multiple .zip formats: I've found macOS Archive Utility sometimes refuses to extract early pkzip .zips created on MS-DOS, but yet Info-ZIP handles them just fine. And, the macOS Archive Utility will complain that a proper .tar.bz2 is "corrupt" created using bzip2. In general, be liberal in input and be conservative in output. Sometimes, this means using less features or certain older formats so that all/mo…

> In general, be liberal in input and be conservative in output.

That is a dangerous maxim in a world with malicious players. In fact this PyPI problem is precisely because zip files are being too readily accepted, even if they have ambiguous meaning. Their fix is (very sensibly) to be less liberal with their input.

Re: Preventing ZIP parser confusion attacks on Python package installers

#16

Post by burnt-resistor »

quietbritishjim wrote: post #15
burnt-resistor wrote: post #14

Related to multiple .zip formats: I've found macOS Archive Utility sometimes refuses to extract early pkzip .zips created on MS-DOS, but yet Info-ZIP handles them just fine. And, the macOS Archive Utility will complain that a proper .tar.bz2 is "corrupt" created using bzip2. In general, be liberal in input and be conservative in output. Sometimes, this means using less features or certain older formats so that all/mo…

> In general, be liberal in input and be conservative in output. That is a dangerous maxim in a world with malicious players. In fact this PyPI problem is precisely because zip files are being too readily accepted, even if they have ambiguous meaning. Their fix is (very sensibly) to be less liberal with their input.

No, it's a well-regarded, fundamental engineering principle of standard and interoperable systems.

Re: Preventing ZIP parser confusion attacks on Python package installers

#17

Post by quietbritishjim »

burnt-resistor wrote: post #16
quietbritishjim wrote: post #15

Earlier quoted context omitted.

> In general, be liberal in input and be conservative in output. That is a dangerous maxim in a world with malicious players. In fact this PyPI problem is precisely because zip files are being too readily accepted, even if they have ambiguous meaning. Their fix is (very sensibly) to be less liberal with their input.

No, it's a well-regarded, fundamental engineering principle of standard and interoperable systems.

Maybe in the early days of the internet, or in closed systems, but in the open internet it's naive and actually leads to more brittle systems.

Sorry to repeat myself, but case in point is the article we're discussing! If something is accepted that is not in the specification then obviously its behaviour is unspecified. That means different implementations can easily have different behaviours, which can lead to security issues exactly like this one.

Even when there's no malice involved, it can often lead to de facto extensions to the specification. If several implementations accept something outside the standard (with roughly the same behaviour), and one accidentally produces it, then soon it becomes relied upon and all implementations need to handle it. Then the standard is no longer authoritative, or has to be retrospectively updated (see for example the huge part of the HTML spec that deals with otherwise invalid HTML).

Re: Preventing ZIP parser confusion attacks on Python package installers

#18

Post by woodruffw »

burnt-resistor wrote: post #16
quietbritishjim wrote: post #15

Earlier quoted context omitted.

> In general, be liberal in input and be conservative in output. That is a dangerous maxim in a world with malicious players. In fact this PyPI problem is precisely because zip files are being too readily accepted, even if they have ambiguous meaning. Their fix is (very sensibly) to be less liberal with their input.

No, it's a well-regarded, fundamental engineering principle of standard and interoperable systems.

This hasn't been considered well-regarded for a very long time. I'd say the opposite is dogmatic: we're in a post-Postel world[1], in part because of observed security failures over the last 30 years.

[1]: https://alexgaynor.net/2025/mar/25/postels-law-and-the-three...

Post reply on HN