Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Xz/liblzma: Bash-stage Obfuscation Explained

gynvael.coldwind.pl

Post reply on HN
Print view View original

61–70 of 137 posts

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#61

Post by chii »

Martinussen wrote: post #39
plg94 wrote: post #17

Earlier quoted context omitted.

the bad actor was a co-maintainer of the repo (and even more active than the original maintainer for quite some time) with full commit rights. This was strait committed to master, no PR and no review required. edit: also this was heavily obfuscated in some binary files that were marked as test files ("good" and "bad" xz compressed test file). No way to spot this if you don't know what you're looking for.

> No way to spot this if you don't know what you're looking for. I would expect most people to at least ask for more clarification on random changes to `head` offsets, honestly - or any other diff there. If they had access to just merge whatever with no oversight, I guess the blame is more on people using this in other projects without vetting their basic security of projects they fully, implicitly trust, though. As…

> without vetting their basic security of projects they fully

this sort of vetting you're talking about is gonna turn up nothing. Most vetting is at the source code level anyway, not in the tests, nor the build files tbh. It's almost like a checkbox "cover your ass" type work that a hired consultant would do.

Unless you're someone in gov't/military, in which case yes, you'd vet the code deeply. But that costs an arm and a leg. Would a datacenter/hosting company running ssh servers do that?

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#62

Post by mrb »

SV_BubbleTime wrote: post #46
mrb wrote: post #32

Earlier quoted context omitted.

To be clear: the build system did not use the code fragment you quoted. This complex awk code is a later stage of the backdoor.

I see, my point was more than this shouldn’t be allowed. I think part of the problem with a lot of things is we’re allowing complexity for the sake of complexity. No one has simplicity-required checks. My previous post should say “allows things like this”.

But what are you suggesting exactly? The code fragment you quoted was awk code. Awk is a generic programming language. Any programming language can be written to be complex and unreadable.

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#63

Post by chii »

mrkramer wrote: post #49
dboreham wrote: post #47

Earlier quoted context omitted.

> verify the identity of open source maintainer/s and question their motive for joining the open source project. This kind of goes against the whole "free" thing.

Anybody is free to contribute if s/he is contributing in a good will but what happens if you don't know who they are and what are their motives? You can look at the their track record for example, that's another way to determining their credibility. In another words you need to establish trust somehow. Idk if this specific individual that backdoored XZ had a track record of contributing to other open source projects(…

Just because you know the identity of an individual, doesn't mean they are trustworthy. They might be compromised, or they might be willfully doing it for their own personal gain, regardless of their existing reputation (or even, leveraging their existing reputation - bernie madoff was a well known and well respected investment banker).

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#64

Post by fullstop »

zzzeek wrote: post #57
fullstop wrote: post #52

Earlier quoted context omitted.

It really wasn't, though. commit 74b138d2a6529f2c07729d7c77b1725a8e8b16f1 Author: Jia Tan Date: Sat Mar 9 10:18:29 2024 +0800 Tests: Update two test files. The original files were generated with random local to my machine. To better reproduce these files in the future, a constant seed was used to recreate these files. diff --git a/tests/files/bad-3-corrupt_lzma2.xz b/tests/files/bad-3-corrupt_lzma2.xz index 926f95b0.…

> Would you bat an eye at this? If it were from a trusted developer and the code was part of a test case? well lets all agree that now, if we see commits affecting / adding binary data with "this was generated locally with XYZ", that now we will bat an eye at it.

Without a doubt!

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#65

Post by minimax »

> if this was found by accident, how many things still remain undiscovered.

This, to me, is the most important question. There is no way Andres Freund just happened to find the _only_ backdoored popular open source project out there. There must be like a dozen of these things in the wild?

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#66

Post by oogali »

vodou wrote: post #41

A disturbing thought here is that unit tests opened up an attack vector. Without the tests this would have been much harder to hide.

Furthermore, the attacker covered their tracks on the initial payload with an innocuous paragraph in the README. ("Nothing to see here!")

    bad-3-corrupt_lzma2.xz has three Streams in it. The first and third
    streams are valid xz Streams. The middle Stream has a correct Stream
    Header, Block Header, Index and Stream Footer. Only the LZMA2 data
    is corrupt. This file should decompress if --single-stream is used.
The strings of `####Hello####` and `####World####` are there so that if you actually follow the instructions in the README, you get a seemingly valid result.

    $ cat tests/files/bad-3-corrupt_lzma2.xz | xz -d --single-stream
    ####Hello####
They're shell comments so it won't interfere with payload execution.

And lastly, they act as a marker that can be used by a later regex to locate the file _without_ referencing it by name directly nor using the actual Hello and World strings.

    $ gl_am_configmake=`grep -aErls "#{4}[[:alnum:]]{5}#{4}$" $srcdir/ 2>/dev/null`
    $ echo $gl_am_configmake
    ./tests/files/bad-3-corrupt_lzma2.xz

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#67

Post by johnisgood »

fullstop wrote: post #52
johnisgood wrote: post #43

Earlier quoted context omitted.

> It was never committed in plain sight. It was though. I have seen those two test files being added by a commit on GitHub. Unfortunately it has been disabled by now, so I cannot give you a working link.

It really wasn't, though. commit 74b138d2a6529f2c07729d7c77b1725a8e8b16f1 Author: Jia Tan Date: Sat Mar 9 10:18:29 2024 +0800 Tests: Update two test files. The original files were generated with random local to my machine. To better reproduce these files in the future, a constant seed was used to recreate these files. diff --git a/tests/files/bad-3-corrupt_lzma2.xz b/tests/files/bad-3-corrupt_lzma2.xz index 926f95b0.…

Yeah, again, "committed in plain sight" it was, was it not? Batting an eye on it or not is another matter.

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#68

Post by swells34 »

SV_BubbleTime wrote: post #46
mrb wrote: post #32

Earlier quoted context omitted.

To be clear: the build system did not use the code fragment you quoted. This complex awk code is a later stage of the backdoor.

I see, my point was more than this shouldn’t be allowed. I think part of the problem with a lot of things is we’re allowing complexity for the sake of complexity. No one has simplicity-required checks. My previous post should say “allows things like this”.

Your point is likely entirely valid, but the example you used is the wrong one.

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#69

Post by MPSimmons »

notnmeyer wrote: post #60

i don’t have a better answer, but this convoluted mess of bash is a smell isn’t it? i live in a different part of the dev world, but could this be written to be less obtuse so it’s more obvious what’s happening? i get that a maintainer can still get malicious code in without the same rigor as an unaffiliated contributor, but surely there’s a better way than piles of “concise” (inadvertently obfuscated?) code?

>this convoluted mess of bash is a smell isn’t it

At a glance, I don't think so. At least, not the fact that the bash looks like a convoluted mess. Sometimes that's how a tight bash script looks (whether it SHOULD be tight or not is a different argument).

For me, the thing that looks suspicious is the repeated tr calls. If I see that, I assume someone is trying to be clever, where 'clever' here is a pejorative. If I were a maintainer and someone checked that in, I'd ask them to walk me through what it was doing, because there's almost always a better solution to avoid chaining things like that.

The real problem here is that there wasn't another maintainer to look at this code being brought in. A critical piece of the stack relied on a single person who, in this case, was malicious.

Re: Xz/liblzma: Bash-stage Obfuscation Explained

#70

Post by duped »

notnmeyer wrote: post #60

i don’t have a better answer, but this convoluted mess of bash is a smell isn’t it? i live in a different part of the dev world, but could this be written to be less obtuse so it’s more obvious what’s happening? i get that a maintainer can still get malicious code in without the same rigor as an unaffiliated contributor, but surely there’s a better way than piles of “concise” (inadvertently obfuscated?) code?

The shell is generated, not written. There are mountains of generated shell configuration code out there due to the prevalence of autoconf, which relies on these M4 (a macro preprocessor) scripts to generate thousands of lines of unreadable shell script (which has to be portable).

This is how a non negligible number of your tools are built.

Post reply on HN