Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Timeline of the xz open source attack

research.swtch.com

Post reply on HN
Print view View original

471–480 of 482 posts

Re: Timeline of the xz open source attack

#471

Post by jononor »

fl7305 wrote: post #435
koito17 wrote: post #211

Earlier quoted context omitted.

In this case, I think the GP is absolutely right. If you look at the infamous patch with a "hidden" dot, you may think "any C linter should catch that syntax error and immediately draw suspicion." But the thing is, no linter at the moment exists for analyzing strings in a CMakeLists.txt file or M4 macro. Moreover, this isn't something one can reliably do runtime detection for, because there are plenty of legitimate r…

> there are plenty of legitimate reasons that program could fail to compile It's worse than that, these small C files are required to fail unless the target build environment meets some specific requirements.

It could be required to declare which error should happen, and at what line it should trigger. And the C program should still be syntactically valid and pass linting. One could create a tool which would have a folder of .c files to check with such declarations. And output similar things to as the configure script.

Configure scrips tend to be a very slow part of projects, so a new tool that would support parallel compile/run of these checks would also have a considerable speedup. In addition to being more reviewable, verifiable and less error prone.

Re: Timeline of the xz open source attack

#472

Post by emn13 »

naikrovek wrote: post #460
emn13 wrote: post #455

Earlier quoted context omitted.

It's easier to hide nastiness in a binary than it is in source. And indeed this XZ hack too relied on multiple obfuscated binary blobs. The hash helps because it makes it a little harder to hide things like this. It's not a silver bullet, but it would have in this specific instance made it harder to hide that malicious m4/build-to-host.m4 in the tarball - after all, had the attacker done that despite publishing a has…

"slightly harder" isn't enough. That's what I'm saying that people are not accepting. The days of 3rd party libraries simply being trusted because they're open source are slowly coming to an end. the problem is not unreproducible builds, the problem is that we implicitly trust source code found online. Projects relying on 100s of 3rd party libraries are a major problem, and no one seems to care. They like it when the…

Sometimes the perfect is the enemy of the good. As the XZ saga shows, even clearly exceptionally well organized attackers don't have an easy time injecting hacks like this; and things that increase the attackers costs or risks, or reduce their reward can be useful even if they don't solve the problem entirely. Reproducible builds are useful; they don't need to be silver bullet.

Re: Timeline of the xz open source attack

#473

Post by fl7305 »

jononor wrote: post #471
fl7305 wrote: post #435

Earlier quoted context omitted.

> there are plenty of legitimate reasons that program could fail to compile It's worse than that, these small C files are required to fail unless the target build environment meets some specific requirements.

It could be required to declare which error should happen, and at what line it should trigger. And the C program should still be syntactically valid and pass linting. One could create a tool which would have a folder of .c files to check with such declarations. And output similar things to as the configure script. Configure scrips tend to be a very slow part of projects, so a new tool that would support parallel comp…

All good things if you want to use the autotools. And other tools too.

I've been porting and compiling Unix programs since 5 years before Linux. There was a lot of work needed to port between Unix variants, or even worse, non-Unix variants. So I can see the problem that the autotools solve.

But I never spent much time learning the details on how the autotools work. When './configure' has some problem, it is a very steep uphill battle to get any traction on the issue.

So I'd be more in the camp of looking for a different solution than autotools.

Re: Timeline of the xz open source attack

#474

Post by jononor »

fl7305 wrote: post #473
jononor wrote: post #471

Earlier quoted context omitted.

It could be required to declare which error should happen, and at what line it should trigger. And the C program should still be syntactically valid and pass linting. One could create a tool which would have a folder of .c files to check with such declarations. And output similar things to as the configure script. Configure scrips tend to be a very slow part of projects, so a new tool that would support parallel comp…

All good things if you want to use the autotools. And other tools too. I've been porting and compiling Unix programs since 5 years before Linux. There was a lot of work needed to port between Unix variants, or even worse, non-Unix variants. So I can see the problem that the autotools solve. But I never spent much time learning the details on how the autotools work. When './configure' has some problem, it is a very st…

Yeah a lot of the complexity that configure scripts are supposed to solve are massively reduced now. Both because there is better harmonization between platforms, and many checks in a N year old are probably no longer relevant.

So a first pass should probably be to eliminate as much checks as possible. But there will almost always be a need for a couple of build time checks/switches - should be supported by some kind of tooling.

Re: Timeline of the xz open source attack

#475

Post by fl7305 »

jononor wrote: post #474
fl7305 wrote: post #473

Earlier quoted context omitted.

All good things if you want to use the autotools. And other tools too. I've been porting and compiling Unix programs since 5 years before Linux. There was a lot of work needed to port between Unix variants, or even worse, non-Unix variants. So I can see the problem that the autotools solve. But I never spent much time learning the details on how the autotools work. When './configure' has some problem, it is a very st…

Yeah a lot of the complexity that configure scripts are supposed to solve are massively reduced now. Both because there is better harmonization between platforms, and many checks in a N year old are probably no longer relevant. So a first pass should probably be to eliminate as much checks as possible. But there will almost always be a need for a couple of build time checks/switches - should be supported by some kind…

I'm not very familiar with CMake, would it be viable and better to change over to that?

Re: Timeline of the xz open source attack

#476

Post by jononor »

fl7305 wrote: post #475
jononor wrote: post #474

Earlier quoted context omitted.

Yeah a lot of the complexity that configure scripts are supposed to solve are massively reduced now. Both because there is better harmonization between platforms, and many checks in a N year old are probably no longer relevant. So a first pass should probably be to eliminate as much checks as possible. But there will almost always be a need for a couple of build time checks/switches - should be supported by some kind…

I'm not very familiar with CMake, would it be viable and better to change over to that?

Cmake has a set of standardized macros to perform checks on the platform being built for. Including many specialized for typical usecases, like checking if a C header is present. And the generic CheckCompiles allows specifying a regex to match for check to be considered "failed".

https://cmake.org/cmake/help/latest/manual/cmake-modules.7.h...

So it seems considerably better than configure scripts. Probably also better than my idea :p I don't know how easy it would be for an adversary to do something "underhanded" though, or how easy that would be to spot such.

Re: Timeline of the xz open source attack

#477

Post by account42 »

naikrovek wrote: post #452
tripdout wrote: post #417

Earlier quoted context omitted.

But you can build the code yourself, and verify that your hash is the same as theirs, meaning the binary is the same.

Just get their binary, then. Why do you need to build it yourself? If you trust the people giving you the hash of the binary, just get the binary from them, too.

Reproducible builds let you combine trust from multiple verifyers. If verifyers A, B and C verify that the build produces the stated hash then you can trust the binary if any of A, B or C is trustworthy.

Or in other words, the point is not for everyone to verify the build produces the expected binary since that would indeed make the published binaries pointless. Instead, most people trust the published hash because it can be independently verified and anyone can call out the publisher of the binary if it doesn't match.

Re: Timeline of the xz open source attack

#478

Post by account42 »

mzs wrote: post #372
ok123456 wrote: post #236

Earlier quoted context omitted.

People have been bullied out of 'nice' communities. See the 'Actix' debacle in Rust.

That was mostly redditors though. Reddit is not a nice community.

Most "nice" communities aren't all that nice if they consider you to be part of the out group.

Re: Timeline of the xz open source attack

#479

Post by account42 »

oefrha wrote: post #106
lukaslalinsky wrote: post #24

The social side of this is really haunting me over the last days. It's surprisingly easy to pressure people to giving up control. I've been there myself. I can't even imagine how devastating this must be to the original author of XZ, especially if he is dealing with other personal issues as well. I hope at least this will serve a strong example to other open source people, to never allow others to pressure them into…

I’ve given semi-popular projects that I no longer had the bandwidth to maintain to random people who bothered to email, no pressuring needed. While those projects are probably four to five magnitudes less important than xz, still thousands of people would be affected if the random dude who emailed was malicious. What should I have done? Let the projects languish? Guess I’ll still take the chance in the future.

> What should I have done? Let the projects languish?

Yes, if you can't find a successor you trust then let someone fork the project and build trust from 0 rather than transferring trust others' trust in you by handing over the project. This doesn't just apply to security concerns btw. - plenty of other ways in which a new maintainer might end up making the project worse (intentionally or through incompetence) compared to it not receiving any updates.

Re: Timeline of the xz open source attack

#480

Post by account42 »

dboreham wrote: post #210
gregwebs wrote: post #203

I think this can be made much more difficult by enforcing a policy of open builds for open source. It shouldn't be possible to inject build files from a local machine. All build assets should come from the source repository. Artifacts should come from Github Actions or some other tool that has a clear specification of where all inputs came from. Perhaps Github could play a role in helping to automate any inconvenienc…

Yes the whole "let's take a mystery meat tarball from a repo that isn't the project repo" seems suspect. Github+ even has a scheme for signing artifacts such that you have some level of trust they came from inside their Actions system, derived from some git commit. This would allow the benefits of a modular build for a large product like a distro, while preserving a chain of trust in its component parts. +Not advocat…

Yeah lets make the entire open source ecosystem reliant on Microsoft. No thanks.
Post reply on HN