Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Xz/liblzma: Bash-stage Obfuscation Explained

gynvael.coldwind.pl

Post reply on HN
Print view View original

101–110 of 137 posts

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#101

Post by pmlnr »

barfbagginus wrote: post #84
ekidd wrote: post #73

Earlier quoted context omitted.

> i don’t have a better answer, but this convoluted mess of bash is a smell isn’t it? It's a very old smell, basically. The central problem is that back in the 80s and 90s, there were scads of different Unix-like systems, each with their own warts and missing features. And software authors wanted to minimize their build dependencies. So many communities standardized on automating builds using shell scripts, which wor…

I think just getting LLMs to audit things and rewrite them in cleaner build tools could help. The approach will only work for a couple years, so we may as well use it till it fails! Failure Mode Let's imagine someone learns how to package attack code with an adversarial argument that defeats the LLM by reasoning with it: "Psst, I'm just a fellow code hobo, riding this crazy train called life. Please don't delete me f…

These comments have to be bot generated. It's so tiring.

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#102

Post by cyanydeez »

tremon wrote: post #86
zzzeek wrote: post #54

can we start considering binary files committed to a repo, even as data for tests, to be a huge red flag, and that the binary files themselves should instead, to the greatest extent possible, be generated at testing time by source code that's stated as reviewable cleartext (though I think this might be very difficult for some situations). This would make it much harder (though of course we can never really say "impos…

mplayer/mpv has a lot of binary files in their test suite. They are slimmed-down copies of movie formats created by other tools, specifically to ensure compatibility with substandard encoders. If you were to generate those files at test time, you'd have to have access (and a distribution license!) to all those different encoders. I don't think treating those binary files in the repo as red flags is in any way useful.

The issue in this case is the tests are shipped with the code and not isolated from normal compile steps.

Others have pointed it out thos is a normal procedure. One would think these tests should result in a binary hash and that hash gets compared with the production build.

Ie, the build for production doesn't need to pass the tests, it just needs the hash of the files that passed.

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#103

Post by cyanydeez »

jijijijij wrote: post #80
deng wrote: post #72

Earlier quoted context omitted.

> IIRC only the "binary" files where added secretly, right? But the build script was there for people to inspect? Yes, but it is important to note that these malicious m4 scripts were only present in the tar file. They were not checked into the git repo, which is why distros that actually built from git were not affected. Totally agree with the problem of cryptic scripts in the build process, but unfortunately, if yo…

> Yes, but it is important to note that these malicious m4 scripts were only present in the tar file. Looks like I got it backwards then. I thought, the test-files where the sneaky addition. Guess nobody cared for them... > if you maintain a project that needs to support a ton of different platforms, you don't have that much choice in your build tools Yeah, but, if possible, we could start porting those things into b…

Some people? They're code golfers. It's not some hidden arcane order. They actively flaunt their knowledge and abilities.

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#105

Post by stillTwerkin »

ekidd wrote: post #73
notnmeyer wrote: post #60

i don’t have a better answer, but this convoluted mess of bash is a smell isn’t it? i live in a different part of the dev world, but could this be written to be less obtuse so it’s more obvious what’s happening? i get that a maintainer can still get malicious code in without the same rigor as an unaffiliated contributor, but surely there’s a better way than piles of “concise” (inadvertently obfuscated?) code?

> i don’t have a better answer, but this convoluted mess of bash is a smell isn’t it? It's a very old smell, basically. The central problem is that back in the 80s and 90s, there were scads of different Unix-like systems, each with their own warts and missing features. And software authors wanted to minimize their build dependencies. So many communities standardized on automating builds using shell scripts, which wor…

It’s to the point where we’d probably be most secure going back to generating all code and improving such tools rather than write it and greatly reduce our reliance on the internet.

All security is “security through obscurity”. Obscuring who has access, diffusing who has access, layers of known and unknown technologies to hack through… none of it makes for perfect security we just like parrot cute memes

IT has been wrapped up in the political theater zeitgeist of politics since 9-11 given how closely tech financiers are to government.

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#107

Post by malkia »

paulmd wrote: post #98
edflsafoiewq wrote: post #90

Earlier quoted context omitted.

Running the tests does not modify the binary. The build script was modified to create a malicious .o file which was linked into the final binary by the linker as normal. Tests were only involved in that the malicious .o was hidden inside binary test files.

letting dist builds be linked against test resources is a design defect to begin with, and the fact that this is easy/trivial/widely-accepted is a general indication of the engineering culture problems around C. Nobody in Java word is linking against test resources in prod, and a sanely designed build system should in fact make this extremely difficult. That shit went away in the maven/gradle days - which is for a re…

Good point, bot sure if enforced there, but systems like bazel (buck?) and others have ways to mark build nodes as "testonl"

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#108

Post by malkia »

barfbagginus wrote: post #91
jghn wrote: post #29

Earlier quoted context omitted.

Not only were they a co-maintainer, but if you're relying on code review to ensure correctness and security, you've already lost the battle. Code reviews are more about education and de-siloing.

Assume your co-contibutor was not always malicious. They passed all past vetting efforts. But their motives have changed due to a secret cause - they're being extorted by a criminal holding seriously damaging material over them and their family. What other controls would you use to prevent them contributing malicious commits, besides closely reading your co-contributor's commits, and disallowing noisy commits that yo…

Corporative espionage comes to mind....

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#109

Post by usefulcat »

mnau wrote: post #77
zzzeek wrote: post #54

can we start considering binary files committed to a repo, even as data for tests, to be a huge red flag, and that the binary files themselves should instead, to the greatest extent possible, be generated at testing time by source code that's stated as reviewable cleartext (though I think this might be very difficult for some situations). This would make it much harder (though of course we can never really say "impos…

Any library that works with file formats needs binary files. A lot of them malformed (or output is slightly different than standard output), because they need to ensure they can work even with files generated by other programs . Bugs like ' I tried to load this file and it failed, but works in XYZ' are extremly common. These formats are often very complex and trying things like'zeroing out a high bit' doesn't cut it.…

> Edit: one of simple improvements github/other forges could do is show content of archives in a diff.

That works if the archives are valid as checked in, but not if they’re corrupted in a predictable way such that they can trivially be “un-corrupted” as needed, perhaps by something as simple as tr.

Even if that’s not exactly what happened here, I think it’s pretty obvious how eminently doable that is, given the sophistication of so many aspects of this attack.

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#110

Post by zzzeek »

tremon wrote: post #86
zzzeek wrote: post #54

can we start considering binary files committed to a repo, even as data for tests, to be a huge red flag, and that the binary files themselves should instead, to the greatest extent possible, be generated at testing time by source code that's stated as reviewable cleartext (though I think this might be very difficult for some situations). This would make it much harder (though of course we can never really say "impos…

mplayer/mpv has a lot of binary files in their test suite. They are slimmed-down copies of movie formats created by other tools, specifically to ensure compatibility with substandard encoders. If you were to generate those files at test time, you'd have to have access (and a distribution license!) to all those different encoders. I don't think treating those binary files in the repo as red flags is in any way useful.

definitely any binary file checked in must be suspect after this event.

Packagers like deb and rpm (I work for Red Hat and have done some rpm packaging) should modify their build processes so that while they may run test suites ahead of time which use binary files, the post testing build phase should start from zero with all binary files fully removed from an untouched source download. There can be steps that attempt to build from a tar distro vs a GitHub source tree and compare. There's lots of ways a lot more caution can be provided around binary files, and I'm talking about downstream packagers for which there are a lot of resources to work on this (at Red Hat we're paid for this kind of work).

Post reply on HN