Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reflections on Distrusting xz

joeyh.name

Post reply on HN
Print view View original

21–30 of 335 posts

Re: Reflections on Distrusting xz

#21

Post by WesolyKubeczek »

ducktective wrote: post #16

I'm here wondering why big tech companies that have Too Much to Lose didn't already massively fund a project that freaking sshd depended on it (through systemd). Like how does it hurt Google to assign 100 people to review and investigate commits of some project as basic and fundamental as a compression tool

Poke at any large company at all, and you'll find that their in-house critical fundamental infrastructure thing is chronically underfunded, understaffed, bug-ridden, everyone is worried but no budget is ever approved.

Re: Reflections on Distrusting xz

#22

Post by funcDropShadow »

JoshTriplett wrote: post #10
fsflover wrote: post #7

> What if xz contains a hidden buffer overflow or other vulnerability, that can be exploited by the xz file it's decompressing? If you generalize this problem further, to all packages, then the only reliable solution is security through compartmentalization. On Qubes OS, any file I open, including .jpg and .avi, can't have the access to my private data or attack the admin account for the whole computer. This is ensur…

> the only reliable solution is security through compartmentalization I hope we get there eventually. Not just for standalone processes, but for individual libraries. A decompression library could run inside a WebAssembly sandbox, with the compressed file as input, the uncompressed file as output, and no other capabilities.

What does this have to do with WebAssembly? That is another runtime that adds complexity. Apple has been sandboxing codecs for a long time. They run in a sandboxed process that is only communicating through stdin and stdout or something similar, if I remember correctly. You can ran native code directly. Adding a runtime with a JIT-compiler makes it harder to understand what is going on.

Re: Reflections on Distrusting xz

#23

Post by antoinealb »

Macha wrote: post #4

Yeah, it's left me a little disappointed in Arch in particular that they didn't follow the lead of Debian and Fedora and revert to a much older version, instead just building 5.6.1 from the git repo and basically defended it as "the hacked build script checked for dpkg/rpm anyway".

Arch Linux is not vulnerable to this specific attack, which requires sshd to be linked to liblzma. This link is provided by out-of-sshd patches, that Arch does not apply to their build.

Re: Reflections on Distrusting xz

#24

Post by rwmj »

gammalost wrote: post #13
berkes wrote: post #2

Full on tinfoil hat here. But warranted and practical. I'm wondering what fallout we'll see from this backdoor in the coming weeks, months or years. Was the backdoor used on obscure build servers or obscure pieces of build infrastructure somewhere? Lying dormant for a moment in future to start injecting code into built packages maybe? Are distro's going to go full-on tinfoil-hat and lock down their distribution, halt…

You can't really remove dependencies in open source. It is so intertwined at this point that doing it would be too expensive for most companies. I think the solution is to containerize, containerize and then containerize some more times and make it all with zero trust in mind.

Containerizing is entirely the worst response here. Containers, as deployed in the real world, are basically massive binary blobs of completely uncertain origin, usually hard to reproduce, that easily permit the addition of unaudited invisible changes.

(Yes yes, I know there are some systems which try to mitigate this, but I say as deployed in the real world.)

Re: Reflections on Distrusting xz

#25

Post by 01HNNWZ0MV43FF »

ducktective wrote: post #16

I'm here wondering why big tech companies that have Too Much to Lose didn't already massively fund a project that freaking sshd depended on it (through systemd). Like how does it hurt Google to assign 100 people to review and investigate commits of some project as basic and fundamental as a compression tool

Very shrimple https://en.wikipedia.org/wiki/Tragedy_of_the_commons

Re: Reflections on Distrusting xz

#26

Post by berkes »

skywhopper wrote: post #11
berkes wrote: post #2

Full on tinfoil hat here. But warranted and practical. I'm wondering what fallout we'll see from this backdoor in the coming weeks, months or years. Was the backdoor used on obscure build servers or obscure pieces of build infrastructure somewhere? Lying dormant for a moment in future to start injecting code into built packages maybe? Are distro's going to go full-on tinfoil-hat and lock down their distribution, halt…

IIRC Debian has wiped and is rebuilding all their build hosts, so yes. But while I understand what you mean, I would not call improving the security of a piece of software “halting progress”. Security improvements are progress, too. Plus, revisiting processes and assumptions can also give opportunities to improve efficiency elsewhere. Maintenance can be an opportunity if you approach it properly.

What I meant with "halting progress" is what commonly happens when a piece of software is "rewritten from scratch". Users (or clients or customers) see no improvements for years or weeks, while the business is burning money like mad.

The main reason why I am firmly opposed to "rewrite from scratch" or "we'll need some weeks to refactor¹".

Removing upstream dependencies and replacing them with other deps, with no-code, or with self-written code² is a task that takes long time during which stakeholders see no value added other than "we reduced the risk"; in case of e.g. SAAS that's not even risk these stakeholders are exposed to, so they then see "nothing improving". I'm certain a lot of managers, CTOs and developers suddenly realize that, wow, dependencies really are a liability.

¹ I am not against refactoring, just very much against refactoring as standalone, large task. Sometimes it's unavoidable because of poor/hard choices made in the past, but it's always a bad option. The good option would be "refactor on touch" - refactoring as part of our daily jobs of writing software.

² Too often do I see dependencies that are redicoulously simple. Left-pad being the posterchild. Or dependencies that bring everything and the kitchen-sink, but all we use is this one tiny bit that would've cost us less than 100 lines to write. Or dependencies to solve -- nothing really? Just that no-one took the time to go through it and remove it. And so forth and so on.

Re: Reflections on Distrusting xz

#27

Post by moomin »

> Hopefully, Lasse Collin will consider these possibilities and address them in his response to the attack.

Here's the thing: Lasse Collin was overloaded back in 2021. I've no particular reason to believe that isn't still the case. He needs help. Dealing with this solo is an incredible amount of work. Also, he needs help from a verifiably trustworthy source, verifiable in a way that doesn't require a lot of effort. In practice, that almost certainly means help from a major open source company.

I seriously doubt that's going to happen, because the people who really need to learn this lesson won't, because it's probably not in their financial interest to realize that supply chain problems start with them doing things on the cheap. While we keep on running the world's infrastructure like XKCD 2347 every so often everything's going to topple over.

Re: Reflections on Distrusting xz

#29

Post by 01HNNWZ0MV43FF »

funcDropShadow wrote: post #22
JoshTriplett wrote: post #10

Earlier quoted context omitted.

> the only reliable solution is security through compartmentalization I hope we get there eventually. Not just for standalone processes, but for individual libraries. A decompression library could run inside a WebAssembly sandbox, with the compressed file as input, the uncompressed file as output, and no other capabilities.

What does this have to do with WebAssembly? That is another runtime that adds complexity. Apple has been sandboxing codecs for a long time. They run in a sandboxed process that is only communicating through stdin and stdout or something similar, if I remember correctly. You can ran native code directly. Adding a runtime with a JIT-compiler makes it harder to understand what is going on.

There's AOT wasm too. Firefox uses it to sandbox some stuff. https://hacks.mozilla.org/2021/12/webassembly-and-back-again...

Re: Reflections on Distrusting xz

#30

Post by diggan »

Macha wrote: post #4

Yeah, it's left me a little disappointed in Arch in particular that they didn't follow the lead of Debian and Fedora and revert to a much older version, instead just building 5.6.1 from the git repo and basically defended it as "the hacked build script checked for dpkg/rpm anyway".

Is this what you're referring to?

> Regarding sshd authentication bypass/code execution

> Arch does not directly link openssh to liblzma, and thus this attack vector is not possible. You can confirm this by issuing the following command:

> However, out of an abundance of caution, we advise users to remove the malicious code from their system by upgrading either way. This is because other yet-to-be discovered methods to exploit the backdoor could exist.

https://archlinux.org/news/the-xz-package-has-been-backdoore...

I'm not finding anything from Arch mentioning dpkg/rpm, the linked article above is the latest article about the xz compromise from the Arch homepage.

Post reply on HN