Live data from Hacker News

Backdoor in upstream xz/liblzma leading to SSH server compromise

openwall.com

211–220 of 1001 posts

Re: Backdoor in upstream xz/liblzma leading to SSH server compromise

#211
post #99
post #48

For those panicking, here are some key things to look for, based on the writeup: - A very recent version of liblzma5 - 5.6.0 or 5.6.1. This was added in the last month or so. If you're not on a rolling release distro, your version is probably older. - A debian or RPM based distro of Linux on x86_64. In an apparent attempt to make reverse engineering harder, it does not seem to apply when built outside of deb or rpm p…

Ubuntu still ships 5.4.5 on 24.03 (atm). I did a quick diff of the source (.orig file from packages.ubuntu.com) and the content mostly matched the 5.4.5 github tag except for Changelog and some translation files. It does match the tarball content, though. So for 5.4.5 the tagged release and download on github differ. It does change format strings, e.g. +#: src/xz/args.c:735 +#, fuzzy +#| msgid "%s: With --format=raw,…

RHEL9 is shipping 5.2.5; RHEL8 is on 5.2.4.

Re: Backdoor in upstream xz/liblzma leading to SSH server compromise

#212
post #99
post #48

For those panicking, here are some key things to look for, based on the writeup: - A very recent version of liblzma5 - 5.6.0 or 5.6.1. This was added in the last month or so. If you're not on a rolling release distro, your version is probably older. - A debian or RPM based distro of Linux on x86_64. In an apparent attempt to make reverse engineering harder, it does not seem to apply when built outside of deb or rpm p…

Ubuntu still ships 5.4.5 on 24.03 (atm). I did a quick diff of the source (.orig file from packages.ubuntu.com) and the content mostly matched the 5.4.5 github tag except for Changelog and some translation files. It does match the tarball content, though. So for 5.4.5 the tagged release and download on github differ. It does change format strings, e.g. +#: src/xz/args.c:735 +#, fuzzy +#| msgid "%s: With --format=raw,…

FYI, your formatting is broken. Hacker News doesn't support backtick code blocks, you have to indent code.

Anyway, so... the xz project has been compromised for a long time, at least since 5.4.5. I see that this JiaT75 guy has been the primary guy in charge of at least the GitHub releases for years. Should we view all releases after he got involved as probably compromised?

Re: Backdoor in upstream xz/liblzma leading to SSH server compromise

#213

I am *not* a security researcher, nor a reverse engineer. There's lots of stuff I have not analyzed and most of what I observed is purely from observation rather than exhaustively analyzing the backdoor code. I love this sort of technical writing from contributors outside the mainstream debugging world who might be averse to sharing. What an excellently summarized report of his findings that should be seen as a templ…

For what it's worth the author is a PostgreSQL committer, he's not a security researcher but he's a pretty damn good engineer!

Re: Backdoor in upstream xz/liblzma leading to SSH server compromise

#214

Yikes! Do you have any info on the individual's background or possible motivations?

I would presume it's a state actor. Generally in the blackhat world, attackers have very precise targets. They want to attack this company or this group of individuals. But someone who backdoors such a core piece of open source infrastructure wants to cast a wide net to attack as many as possible. So that fits the profile of a government intelligence agency who is interested in surveilling, well, everything.

Or it could in theory be malware authors (ransomware, etc). However these guys tend to aim at the low hanging fruits. They want to make a buck quickly. I don't think they have the patience and persistence to infiltrate an open source project for 2 long years to finally gain enough trust and access to backdoor it. On the other hand, a state actor is in for the long term, so they would spend that much time (and more) to accomplish that.

So that's my guess: Jia Tan is an employee of some intelligence agency. He chose to present an asian persona, but that's not necessarily who he truly represents. Could be anyone, really: Russia, China, Israel, or even the US, etc.

Edit: given that Lasse Collin was the only maintainer of xz utils in 2022 before Jia Tan, I wouldn't be surprised if the state actor interfered with Lasse somehow. They could have done anything to distract him from the project: introduce a mistress in his life, give him a high-paying job, make his spouse sick so he has to care for her, etc. With Lasse not having as many hours to spend on the project, he would have been more likely to give access to a developer who shows up around the same time and who is highly motivated to contribute code. I would be interested to talk to Lasse to understand his circumstances around 2022.

Re: Backdoor in upstream xz/liblzma leading to SSH server compromise

#215
post #106

The discussion to upload it to Debian is interesting on its own https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1067708

That name jumped out at me, Hans Jansen is the name Dominic Monaghan used when posing as a German interviewer with Elijah Woods. Not that it can't be a real person https://youtu.be/IfhMILe8C84

See comments about "Hans Janson" upthread, he appeared to collaborate on the exploit in other ways as well.

Re: Backdoor in upstream xz/liblzma leading to SSH server compromise

#216
post #17

That's completely crazy, the backdoor is introduced through a very cryptic addition to the configure script. Just looking at the diff, it doesn't look malicious at all, it looks like build script gibberish.

Yeah, now imagine they succeeded and it didn't cause any performance issues... Can we even be sure no such successful attempt has already been made?

You can be certain it has happened, many times. Now think of all the software we mindlessly consume via docker, language package managers, and the like.

Remember, there is no such thing as computer security. Make your decisions accordingly :)

Re: Backdoor in upstream xz/liblzma leading to SSH server compromise

#217

A lot of eyes will be dissecting this specific exploit, and investigating this specific account, but how can we find the same kind of attack in a general way if it’s being used in other projects and using other contributor names?

1. Everything must be visible. A diff between the release tarball and tag should be unacceptable. It was hidden from the eyes to begin with.

2. Build systems should be simple and obvious. Potentially not even code. The inclusion was well hidden.

3. This was caught through runtime inspection. It should be possible to halt any Linux system at runtime, load debug symbols and map _everything_ back to the source code. If something can't map back then regard it as a potentially malicious blackbox.

There has been a strong focus and joint effort to make distributions reproducible. What we haven't managed though is prove that the project compromises only of freshly compiled content. Sorta like a build time / runtime "libre" proof.

This should exist for good debugging anyway.

It wouldn't hinder source code based backdoors or malicious vulnerable code. But it would detect a backdoor like this one.

Just an initial thought though, and probably hard to do, but not impossibly hard, especially for a default server environment.

Re: Backdoor in upstream xz/liblzma leading to SSH server compromise

#218
post #98

Earlier quoted context omitted.

So many malicious actors have been caught because they accidentally created a mild annoyance for someone that went on to bird-dog the problem.

Which is why a really good backdoor is a one line logic bug somewhere which is fiendishly difficult to trigger.

[deleted]

Re: Backdoor in upstream xz/liblzma leading to SSH server compromise

#219

That's completely crazy, the backdoor is introduced through a very cryptic addition to the configure script. Just looking at the diff, it doesn't look malicious at all, it looks like build script gibberish.

A big part of the problem is all the tooling around git (like the default github UI) which hides diffs for binary files like these pseudo-"test" files. Makes them an ideal place to hide exploit data since comparatively few people would bother opening a hex editor manually.

00011900: 0000 4883 f804 7416 b85f 5f5f 5f33 010f ..H...t..____3.. │ 00011910: b651 0483 f25a 09c2 0f84 5903 0000 488d .Q...Z....Y...H. │ 00011920: 7c24 40e8 5875 0000 488b 4c24 4848 3b4c |$@.Xu..H.L$HH;L │ 00011930: 2440 7516 4885 c074 114d 85ff 0f84 3202 $@u.H..t.M....2. │ 00011940: 0000 498b 0ee9 2c02 0000 b9fe ffff ff45 ..I...,........E │ 00011950: 31f6 4885 db74 0289 0b48 8bbc 2470 1300 1.H..t...H..$p.. │ 00011960: 0048 85ff 0f85 c200 0000 0f57 c00f 2945 .H.........W..)E │ 00011970: 0048 89ac 2470 1300 0048 8bbc 2410 0300 .H..$p...H..$... │ 00011980: 0048 8d84 2428 0300 0048 39c7 7405 e8ad .H..$(...H9.t... │ 00011990: e6ff ff48 8bbc 24d8 0200 0048 8d84 24f0 ...H..$....H..$. │ 000119a0: 0200 0048 39c7 7405 e893 e6ff ff48 8bbc ...H9.t......H.. │ 000119b0: 2480 0200 0048 8d84 2498 0200 0048 39c7 $....H..$....H9. │ 000119c0: 7405 e879 e6ff ff48 8bbc 2468 0100 004c t..y...H..$h...L │ Please tell me what this code does, Sheldon

Re: Backdoor in upstream xz/liblzma leading to SSH server compromise

#220
post #91

That's completely crazy, the backdoor is introduced through a very cryptic addition to the configure script. Just looking at the diff, it doesn't look malicious at all, it looks like build script gibberish.

The use of "eval" stands out, or at least it should stand out – but there are two more instances of it in the same script, which presumably are not used maliciously. A while back there was a discussion[0] of an arbitrary code execution vulnerability in exiftool which was also the result of "eval". Avoiding casual use of this overpowered footgun might make it easier to spot malicious backdoors. Usually there is a bett…

Unfortunately eval in a shell script has an effect on the semantics but is not necessary to do some kind of parsing of the contents of a variable, unlike Python or Perl or JavaScript. A

    $goo
line (without quotes) will already do word splitting, though it won't do another layer of variable expansion and unquoting, for which you'll need

    eval "$goo"
(This time with quotes).
Post reply on HN