Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Could the XZ backdoor been detected with better Git/Deb packaging practices?

optimizedbyotto.com

Post reply on HN
Print view View original

51–60 of 116 posts

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#51

Post by jacquesm »

dijit wrote: post #34
acka wrote: post #31

I believe the XZ compromise partly stemmed from including binary files in what should have remained a source-only project. From what I remember, well-run projects such as those of the GNU project have always required that all binaries—whether executables or embedded data such as test files—be built directly from source, compiling a purpose-built DSL if necessary. This ensures transparency and reproducibility, both of…

thats not the issue, there will always be prebuilt binaries (hell, deb/rpm are prebuilt binaries). The issue for xz was that the build system was not hermetic (and sufficiently audited). Hermitic build environments that can’t fetch random assets are a pain to maintain in this era, but are pretty crucial in stopping an attack of this kind. The other way is reproducible binaries, which is also very difficult. EDIT: Wel…

This is not going to be popular: I think the whole idea that a build system just fetches resources from outside of the build environment is fundamentally broken. It invites all kinds of trouble and makes it next to impossible to really achieve stability and to ensure that all code that is in the build has been verified. Because after you've done it four times the fifth time you won't be looking closely. But if you don't do it automatically but only when you actually need it you will be looking a lot more sharpish at what has changed since you last pulled in the code. Especially for older and stable libraries the consumers should dictate when they upgrade, not some automatic build process. But because we're all conditioned to download stuff because it may have solved some security issue we stopped to think about the security issues associated with just downloading stuff and dumping it into the build process.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#52

Post by TheDong »

jiggawatts wrote: post #50

Something that the XZ back door made me realise is that the fundamental difference between proprietary and open source software is not the price or source availability for most of its users — no not developers! — it is the reputation and protected brand of the former and the anonymity of the latter. We have no clue who “Jia Tan” is, a name certain to be a pseudonym. Nobody has seen his face. He never provided ID to a…

> Something that the XZ back door made me realise is that the fundamental difference between proprietary and open source software is not the price or source availability for most of its users — no not developers! - it is the reputation and protected brand of the former and the anonymity of the latter.

You're making a distinction not between open source and proprietary software but rather between hobbyist and corporate software.

There are open source projects made by companies with no external contributions allowed (sqlite sorta, most of google and amazon's oss projects in practice etc)

There are proprietary software downloads with no name attached, like practically every keygen, game crack, many indie games posted for free download on forums or 4chan, etc etc.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#53

Post by Tuhbrook »

jiggawatts wrote: post #50

Something that the XZ back door made me realise is that the fundamental difference between proprietary and open source software is not the price or source availability for most of its users — no not developers! — it is the reputation and protected brand of the former and the anonymity of the latter. We have no clue who “Jia Tan” is, a name certain to be a pseudonym. Nobody has seen his face. He never provided ID to a…

Dude what did you smoke? Or are you a state actor yourself? 3 sentences in and there is zero logic, 100 fear mongering in your text.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#54

Post by jacquesm »

mananaysiempre wrote: post #47
dijit wrote: post #40

Earlier quoted context omitted.

I thought that the exploit was not injected into the Git repository on GitHub at all, but only in the release tarballs. And that due to how Autoconf & co. work, it is common for tarballs of Autoconf projects to include extra files not in the Git repository (like the configure script). I thought the attacker exploited the fact that differences between the release tarball and the repository were not considered particul…

First of all, even if that were true, that wouldn’t have much to do with hermetic builds as I understand the term. You could take the release tarball and build it on an air-gapped machine, and (assuming the backdoor liked the build environment on the machine) you would get a backdoored artifact. Fetching assets from the Internet (as is fashionable in the JavaScript, Go, Rust, and to some extent Python ecosystems) doe…

Focusing on the technical angle is imo already a step too far. This was first and foremost a social engineering exercise, only secondary a technical one.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#55

Post by kragen »

jacquesm wrote: post #51
dijit wrote: post #34

Earlier quoted context omitted.

thats not the issue, there will always be prebuilt binaries (hell, deb/rpm are prebuilt binaries). The issue for xz was that the build system was not hermetic (and sufficiently audited). Hermitic build environments that can’t fetch random assets are a pain to maintain in this era, but are pretty crucial in stopping an attack of this kind. The other way is reproducible binaries, which is also very difficult. EDIT: Wel…

This is not going to be popular: I think the whole idea that a build system just fetches resources from outside of the build environment is fundamentally broken. It invites all kinds of trouble and makes it next to impossible to really achieve stability and to ensure that all code that is in the build has been verified. Because after you've done it four times the fifth time you won't be looking closely. But if you do…

I am pretty sure Debian Policy agrees with you, although I can't cite chapter and verse. Certainly Nix and Guix agree with you. But that evidently wasn't the problem here.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#57

Post by kragen »

jiggawatts wrote: post #50

Something that the XZ back door made me realise is that the fundamental difference between proprietary and open source software is not the price or source availability for most of its users — no not developers! — it is the reputation and protected brand of the former and the anonymity of the latter. We have no clue who “Jia Tan” is, a name certain to be a pseudonym. Nobody has seen his face. He never provided ID to a…

I think the difference is that the undoubtedly numerous times that this has happened with Microsoft and other proprietary-software vendors, the users weren't in a position to find out.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#58

Post by jiggawatts »

kragen wrote: post #57
jiggawatts wrote: post #50

Something that the XZ back door made me realise is that the fundamental difference between proprietary and open source software is not the price or source availability for most of its users — no not developers! — it is the reputation and protected brand of the former and the anonymity of the latter. We have no clue who “Jia Tan” is, a name certain to be a pseudonym. Nobody has seen his face. He never provided ID to a…

I think the difference is that the undoubtedly numerous times that this has happened with Microsoft and other proprietary-software vendors, the users weren't in a position to find out.

Okay, how, could someone like Jia Tan sneak code into a codebase where commits can only be made by authenticated users with staff accounts on a private network?

Versus… a random email offers to help, someone says “sure!”, and… that’s it. That’s the entire hurdle.

Google did discover a Chinese hacker working for them on the payroll. That kind of thing does occur, but it’s rare.

It’s massively harder and more risky.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#59

Post by jiggawatts »

Tuhbrook wrote: post #53
jiggawatts wrote: post #50

Something that the XZ back door made me realise is that the fundamental difference between proprietary and open source software is not the price or source availability for most of its users — no not developers! — it is the reputation and protected brand of the former and the anonymity of the latter. We have no clue who “Jia Tan” is, a name certain to be a pseudonym. Nobody has seen his face. He never provided ID to a…

Dude what did you smoke? Or are you a state actor yourself? 3 sentences in and there is zero logic, 100 fear mongering in your text.

Says the anonymous green name, missing the point entirely.

Re: Could the XZ backdoor been detected with better Git/Deb packaging practices?

#60

Post by Sophira »

jacquesm wrote: post #51
dijit wrote: post #34

Earlier quoted context omitted.

thats not the issue, there will always be prebuilt binaries (hell, deb/rpm are prebuilt binaries). The issue for xz was that the build system was not hermetic (and sufficiently audited). Hermitic build environments that can’t fetch random assets are a pain to maintain in this era, but are pretty crucial in stopping an attack of this kind. The other way is reproducible binaries, which is also very difficult. EDIT: Wel…

This is not going to be popular: I think the whole idea that a build system just fetches resources from outside of the build environment is fundamentally broken. It invites all kinds of trouble and makes it next to impossible to really achieve stability and to ensure that all code that is in the build has been verified. Because after you've done it four times the fifth time you won't be looking closely. But if you do…

I completely agree with you - I think that automatic downloading of dependencies when building is a bad idea.

However, for the sake of devil's advocacy, I do also want to point out that the first thing a lot of people used to do after downloading and extracting a source tarball was to run "./configure" without even looking at what it is they were executing - even people who (rightly) hate the "curl | bash" combo. You could be running anything.

Being able to verify what it is you're running is vitally important, but in the end it only makes a difference if people take the time to do so. (And running "./configure --help" doesn't count.)

Post reply on HN