A disturbing thought here is that unit tests opened up an attack vector. Without the tests this would have been much harder to hide.
Xz/liblzma: Bash-stage Obfuscation Explained
41–50 of 137 posts
Re: Xz/liblzma: Bash-stage Obfuscation Explained
#42Earlier quoted context omitted.
No, as far as I understand the binary files must be pointed at here: '$gl_am_configmake' ... But I don't see how. This: 'gl_am_configmake=`grep -aErls "#{4}[[:alnum:]]{5}#{4}$" $srcdir/`' seem to match the '####Hello####', but, as far as I can see, that's supposed to be the already converted script?! I presumed the binary files not to contain human readable strings, maybe that's the whole confusion.
Opening bad-3-corrupt_lzma2.xz in an editor reveals it indeed has the string ####Hello####. I don't know enough about lzma compression streams to explain how this appears in the "compressed" version of the payload, but it does.
Thanks. The riddle has been solved :)
Do you have a (safe web view) version of those files? I would like to see what they look like to a casual observer. Judging by the 'tr' assembly command I would expect the bad-3-corrupt_ligma2.xz to be somewhat recognizable as script.
Re: Xz/liblzma: Bash-stage Obfuscation Explained
#43Never allow complexity in code or so-called engineers who ask to merge tons of shitty code. Get rid of that shit and don't trust committers blindly. Anyone who enables this crap is also a liability.
You do realize that "that shit" was part of the obfuscated and xz-compressed backdoor hidden as binary test file, right? It was never committed in plain sight. You can go to https://git.tukaani.org/xz.git and look at the commits yourself – while the commits of the attacker are not prime examples of "good commits", they don't have glaringly obvious red flags either. This backdoor was very sophisticated and well-hidden…
> It was never committed in plain sight.
It was though. I have seen those two test files being added by a commit on GitHub. Unfortunately it has been disabled by now, so I cannot give you a working link.
Re: Xz/liblzma: Bash-stage Obfuscation Explained
#44How are the binary files passed to those stage-0 commands?
The macro defined in build_to_host.m4 is probably called on the tests subdirectory, so it gets these files as a parameter. EDIT: It is called here and will do the extraction of the backdoor when run in the 'tests' subdirectory: https://salsa.debian.org/debian/xz-utils/-/blob/debian/unsta... EDIT2: So it will get the directory as a parameter, the actual file is encoded indirectly here: https://salsa.debian.org/debian/…
Thanks. Yeah, below I learned the '####Hello####' string to be present in the "bad test file" (I haven't seen it myself). I was just not expecting a "binary" file to be basically a text file and thought the `grep` was matching post extraction somehow. That's the root of my confusion. I do understand now where the file gets localized.
IIRC only the "binary" files where added secretly, right? But the build script was there for people to inspect? If so, I have to say, it's not that obfuscated, to someone who actually knew .m4, I guess. At least the grep line should have raised the question of why. I think, part of the problem is normalization of arcane, cryptic scripts in the first place, where people sign off on things they don't fully understand in the moment, since - c'mon - "knowledge" of these old gibberish scripting languages only lives transiently between Google searches and your working memory.
Without looking it up, can you tell me what this does in bash: `echo $(. /tmp/file)` ?
I think, I've seen at least one "xz backdoor detection script" by "someone trusted" in one of the xz threads here, which was at least as cryptic as the .m4 script, containing several `eval`s. I mean, you could probably throw your head onto the keyboard and there is a good chance it's valid bash or regex, or at least common bash can be indiscernible from random gibberish until you manually disassemble it, feeling smuck and dopaminergic. The condensed arcane wizardry around Linux (bash, autotools, CMake, ...) and C (macros, single letter variable culture, ...) is really fun in a way, but it's such a huge vulnerability in itself, before we even talk memory safety.
Re: Xz/liblzma: Bash-stage Obfuscation Explained
#45Earlier quoted context omitted.
the bad actor was a co-maintainer of the repo (and even more active than the original maintainer for quite some time) with full commit rights. This was strait committed to master, no PR and no review required. edit: also this was heavily obfuscated in some binary files that were marked as test files ("good" and "bad" xz compressed test file). No way to spot this if you don't know what you're looking for.
Not only were they a co-maintainer, but if you're relying on code review to ensure correctness and security, you've already lost the battle. Code reviews are more about education and de-siloing.
Yeah, no. Code review isn't going to catch all bugs, but it does catch a ton as long as it's done sincerely and well. You'd have an extremely hard time trying to sneak code with a syntax problem like this into Linux, for example. The community values and rewards nitpickery and fine-toothing, and for good reason.
Re: Xz/liblzma: Bash-stage Obfuscation Explained
#46Earlier quoted context omitted.
In addition… if your build system has things like this as OK: > xz -dc $top_srcdir/tests/files/$p | eval $i | LC_ALL=C sed "s/\(.\)/\1\n/g" | LC_ALL=C awk 'BEGIN{FS="\n";RS="\n";ORS="";m=256;for(i=0;i You should probably expect the potential for abuse? We’re moving towards complexity that is outpacing human ability for any one person to understand, explain, and thus check an entire object. And for what? Build efficie…
To be clear: the build system did not use the code fragment you quoted. This complex awk code is a later stage of the backdoor.
I see, my point was more than this shouldn’t be allowed. I think part of the problem with a lot of things is we’re allowing complexity for the sake of complexity.
No one has simplicity-required checks. My previous post should say “allows things like this”.
Re: Xz/liblzma: Bash-stage Obfuscation Explained
#47The whole XZ drama reminds me of this[1], in another words, verify the identity of open source maintainer/s and question their motive for joining the open source project. Also reminded me of the relevant XKCD meme[2]. Speaking of obfuscation; I'm not a programmer but I did some research in Windows malware RE and what stuck with me is that every code that is obfuscated or every code that is unused is automatically sus…
> verify the identity of open source maintainer/s and question their motive for joining the open source project.
This kind of goes against the whole "free" thing.
Re: Xz/liblzma: Bash-stage Obfuscation Explained
#48[deleted]
Re: Xz/liblzma: Bash-stage Obfuscation Explained
#49The whole XZ drama reminds me of this[1], in another words, verify the identity of open source maintainer/s and question their motive for joining the open source project. Also reminded me of the relevant XKCD meme[2]. Speaking of obfuscation; I'm not a programmer but I did some research in Windows malware RE and what stuck with me is that every code that is obfuscated or every code that is unused is automatically sus…
> verify the identity of open source maintainer/s and question their motive for joining the open source project. This kind of goes against the whole "free" thing.
Anybody is free to contribute if s/he is contributing in a good will but what happens if you don't know who they are and what are their motives? You can look at the their track record for example, that's another way to determining their credibility. In another words you need to establish trust somehow.
Idk if this specific individual that backdoored XZ had a track record of contributing to other open source projects(in a good will) or if s/he just out of the blue starting contributing to this project. I read somewhere that somebody else recommended him or vouched for him. Somebody needs to fill me in with the details.
Re: Xz/liblzma: Bash-stage Obfuscation Explained
#50How on earth did any of this make it through a code review and get merged in? It seems absurdly careless, unless I am missing something.
[deleted]