Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Could the XZ backdoor been detected with better Git/Deb packaging practices?

optimizedbyotto.com

Post reply on HN
Print view View original

11–20 of 116 posts

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#11

Post by kryptiskt »

simoncion wrote: post #7
flerchin wrote: post #5

> As of today only 93% of all Debian source packages are tracked in git on Debian’s GitLab instance at salsa.debian.org. Some key packages such as Coreutils and Bash are not using version control at all This bends my brain a little. I get that they were written before git, but not before the advent of version control.

> I get that they were written before git, but not before the advent of version control. git clone https://git.savannah.gnu.org/git/bash.git git clone https://git.savannah.gnu.org/git/coreutils.git Plug the repo name into https://savannah.gnu.org/git/?group= to get a link to browse the repo.

Look at the commit log in the bash repo. What good does it do if it notionally is version controlled if the commits look like this:

    2025-07-03 Bash-5.3 distribution sources and documentation bash-5.3 Chet Ramey 896 -103357/+174007

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#12

Post by simoncion »

kryptiskt wrote: post #11
simoncion wrote: post #7

Earlier quoted context omitted.

> I get that they were written before git, but not before the advent of version control. git clone https://git.savannah.gnu.org/git/bash.git git clone https://git.savannah.gnu.org/git/coreutils.git Plug the repo name into https://savannah.gnu.org/git/?group= to get a link to browse the repo.

Look at the commit log in the bash repo. What good does it do if it notionally is version controlled if the commits look like this: 2025-07-03 Bash-5.3 distribution sources and documentation bash-5.3 Chet Ramey 896 -103357/+174007

That looks to be the headline for the public release commit. If you'd bothered to look around for a full sixty seconds, you'd have found that the commits tagged with bash-5.3 and bash-5.2 follow that format.

Here are the headlines for a couple of fix commits:

  Bash-5.2 patch 12: fixes for compat mode leaving extglob enabled after command substitution
  Bash-5.2 patch 1: fix crash with unset arrays in arithmetic contexts
It looks like discussion of the patches happens on the mailing list, which is easy to access from the page that brought you to the repo browser.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#14

Post by XorNot »

Groxx wrote: post #8
ottoke wrote: post #2

How did the changes in the binary test files tests/files/bad-3-corrupt_lzma2.xz and tests/files/good-large_compressed.lzma, and the makefile change in m4/build-to-host.m4) manifest to the Debian maintainer? Was there a chance of noticing something odd?

mostly no, from my reading - it was a multi-stage chain of relatively normal looking things that added up to an exploit. helped by the tests involved using compressed data that wasn't human-readable. you can of course come up with ways it could have been caught, but the code doesn't stand out as abnormal in context. that's all that really matters, unless your build system is already rigid enough to prevent it, and ha…

Compression algorithms are deterministic over fixed data though (possibly with some effort).

There's no good reason to have opaque, non generated data in the repository and it should certainly be a red flag going forwards.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#15

Post by Groxx »

XorNot wrote: post #14
Groxx wrote: post #8

Earlier quoted context omitted.

mostly no, from my reading - it was a multi-stage chain of relatively normal looking things that added up to an exploit. helped by the tests involved using compressed data that wasn't human-readable. you can of course come up with ways it could have been caught, but the code doesn't stand out as abnormal in context. that's all that really matters, unless your build system is already rigid enough to prevent it, and ha…

Compression algorithms are deterministic over fixed data though (possibly with some effort). There's no good reason to have opaque, non generated data in the repository and it should certainly be a red flag going forwards.

committed files with carefully crafted bad data is extremely common for testing how your code handles invalid data, especially with regression tests. and lzma absolutely needs to test itself against bad, possibly-malicious data.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#16

Post by jart »

Folks have been ringing the alarm bell for a decade. https://www.nongnu.org/lzip/xz_inadequate.html xz is insane because it appears to be one of the most legitimately dangerous compression formats with the potential to gigafry your data but is exclusively used by literal turbonormies who unironically want to like "shave off a few kilobytes" and basically get oneshotted by it.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#17

Post by secondcoming »

XorNot wrote: post #14
Groxx wrote: post #8

Earlier quoted context omitted.

mostly no, from my reading - it was a multi-stage chain of relatively normal looking things that added up to an exploit. helped by the tests involved using compressed data that wasn't human-readable. you can of course come up with ways it could have been caught, but the code doesn't stand out as abnormal in context. that's all that really matters, unless your build system is already rigid enough to prevent it, and ha…

Compression algorithms are deterministic over fixed data though (possibly with some effort). There's no good reason to have opaque, non generated data in the repository and it should certainly be a red flag going forwards.

There are tons of reasons to have hand-crafted data in a repository.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#18

Post by sanjams »

Groxx wrote: post #8
ottoke wrote: post #2

How did the changes in the binary test files tests/files/bad-3-corrupt_lzma2.xz and tests/files/good-large_compressed.lzma, and the makefile change in m4/build-to-host.m4) manifest to the Debian maintainer? Was there a chance of noticing something odd?

mostly no, from my reading - it was a multi-stage chain of relatively normal looking things that added up to an exploit. helped by the tests involved using compressed data that wasn't human-readable. you can of course come up with ways it could have been caught, but the code doesn't stand out as abnormal in context. that's all that really matters, unless your build system is already rigid enough to prevent it, and ha…

The article references a technical write-up: https://research.swtch.com/xz-script

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#19

Post by sanjams »

Groxx wrote: post #15
XorNot wrote: post #14

Earlier quoted context omitted.

Compression algorithms are deterministic over fixed data though (possibly with some effort). There's no good reason to have opaque, non generated data in the repository and it should certainly be a red flag going forwards.

committed files with carefully crafted bad data is extremely common for testing how your code handles invalid data, especially with regression tests. and lzma absolutely needs to test itself against bad, possibly-malicious data.

I agree, but perhaps OP is suggesting that the hand-crafted data can be generated in a more transparent way. For example, via a script/tool that itself can be reviewed.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#20

Post by 1970-01-01 »

>Can we trust open source software? Yes — and I would argue that we can only trust open source software.

But should we trust it? No!! That's why we're here!

I'm not satisfied with the author's double-standard-conclusion. Trust, but verify does not have some kind of hall pass for OSS "because open-source is clearly better."

Trust, but verify is independent of the license the coders choose.

Post reply on HN