Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

github.com

Post reply on HN
Print view View original

211–220 of 500 posts

Re: Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

#211

Post by IshKebab »

Alifatisk wrote: post #18
asveikau wrote: post #10

It's pretty interesting that they didn't just introduce an RCE that anyone can exploit, it requires the attacker's private key. It's ironically a very security conscious vulnerability.

For real, it's almost like a state-sponsored exploit. It's crafted and executed incredibly well, the performance issue feels like pure luck it got found.

I don't think it was executed incredibly well. There were definitely very clever aspects but they made multiple mistakes - triggering Valgrind, the performance issue, using a `.` to break the Landlock test, not giving the author a proper background identity.

I guess you could also include the fact that they made it a very obvious back door rather than an exploitable bug, but that has the advantage of only letting you exploit it so it was probably an intentional trade-off.

Just think how many back doors / intentional bugs there are that we don't know about because they didn't make any of these mistakes.

Re: Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

#212

Post by mrkramer »

btown wrote: post #163
declan_roberts wrote: post #87

Earlier quoted context omitted.

I agree that this is probably about persistence. Initially I thought the developer was playing the long-con to dump some crypto exchange and make off with literally a billion dollars or more. But if that was the case they wouldn't bother with the key. It'd be a one-and-done situation. It would be a stop-the-world event. Now it looks more like nation-state spycraft.

It's worth also noting that the spycraft involved a coordinated harassment campaign of the original maintainer, with multiple writing styles, to accelerate a transition of maintainership to the attacker: https://www.mail-archive.com/xz-devel@tukaani.org/msg00566.h... https://www.mail-archive.com/xz-devel@tukaani.org/msg00568.h... https://www.mail-archive.com/xz-devel@tukaani.org/msg00569.h... While this doesn't prove…

At first I thought the guy who did this was a lone wolf but now I believe it was indeed state actor. They coordinated and harassed original maintainer into giving them access to the project, basically they hijacked the open source project. The poor guy(the original maintainer) was alone against state actor who was persistent with the goal of hijacking and then backdooring the open source project.

It seems like they were actively looking[0] which open source compression library they can inject with vulnerable code and then exploit and backdoor afterwards.

[0] https://lwn.net/Articles/967763/

Re: Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

#213

Post by takeda »

stingraycharles wrote: post #38
Alifatisk wrote: post #18

Earlier quoted context omitted.

For real, it's almost like a state-sponsored exploit. It's crafted and executed incredibly well, the performance issue feels like pure luck it got found.

I like the theory that actually, it wasn’t luck but was picked up on by detection tools of a large entity (Google / Microsoft / NSA / whatever), and they’re just presenting the story like this to keep their detection methods a secret. It’s what I would do.

Sorry for that ugly comparison, but that explanation reminds me of the theories when covid started, that it was created by secret organization that is actually ruling the world.

People love when there's some explanation that doesn't involve randomness, because with randomness looks like we don't have grasp on things.

Google actually had tooling that was detecting it, but he disabled check that would show it.

Google/Microsoft/NSA could just say they detected it with internal tooling and not disclose how exactly. Google and Microsoft would love to have credit.

Re: Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

#214

Post by bawolff »

declan_roberts wrote: post #77

One thing I notice about state-level espionage and backdoors. The USA seems to have an affinity for hardware interdiction as opposed to software backdoors. Hardware backdoors make sense since much of it passes through the USA. Other countries such as Israel are playing the long-con with very well engineered, multi-year software backdoors. A much harder game to play.

> Other countries such as Israel are playing the long-con with very well engineered, multi-year software backdoors

What is this in reference to?

Re: Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

#215

Post by metzmanj »

jsmith99 wrote: post #52
stingraycharles wrote: post #38

Earlier quoted context omitted.

I like the theory that actually, it wasn’t luck but was picked up on by detection tools of a large entity (Google / Microsoft / NSA / whatever), and they’re just presenting the story like this to keep their detection methods a secret. It’s what I would do.

The attacker changed the projects contact details at oss fuzz (an automated detection tool). There’s an interesting discussion as to whether that would have picked up the vulnerability https://github.com/google/oss-fuzz/issues/11760

I work on oss-fuzz.

I don't think it's plausible OSS-Fuzz could have found this. The backdoor required a build configuration that was not used in OSS-Fuzz.

I'm guessing "Jia Tan" knew this and made changes to XZ's use of OSS-Fuzz for the purposes of cementing their position as the new maintainer of XZ, rather than out of worry OSS-Fuzz would find the backdoor as people have speculated.

Re: Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

#216

Post by jmb99 »

FergusArgyll wrote: post #128
gitfan86 wrote: post #41

Earlier quoted context omitted.

Yeah, but then you would have ssh traffic without a matching login. Wonder if any anomaly detection would work on that

Interesting... Though you can edit whatever log file you want

Any log that root on that box has write access to. It’s theoretically possible to have an anomaly detection service running on a vulnerable machine dumping all of its’ data to an append-only service on some other non-compromised box. In that case, (in this ideal world) the attacker would not be able to disable the detection service before it had logged the anomalous traffic, and wouldn’t be able to purge those logs since they were on another machine.

I’m not aware of any services that a) work like this, or b) would be able to detect this class of attack earlier than last week. If someone does though, please share.

Re: Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

#217

Post by gnramires »

mlyle wrote: post #159
mr_mitm wrote: post #73

Earlier quoted context omitted.

Note that port knocking is vulnerable to replay attacks. Single Packet Authentication is better, but requires a private key (can be your SSH key). https://www.cipherdyne.org/fwknop/

Naive knocking isn't good as a primary security mechanism, but it lowers your attack surface and adds defense in depth. It means that people who can't intercept traffic can't talk to the ssh server-- and that's most attackers at the beginning phases of an attack. And even someone who can intercept traffic needs to wait for actual administrative activity.

Defense in depth has value I agree, but I think it can also be counterproductive in some cases. Every layer can also be buggy and have vulnerabilities, which can often leak (e.g. into code execution) and compromise the whole system (bypassing layers). What happened in this case seems to be a case of maintained hijacking and introducing vulnerabilities. Adding an additional dependency (of say a port-knocking library) doesn't look great in that regard, if the dependency can be hijacked to add remote code execution capabilities. And that library is likely a lot less scrutinized than OpenSSH!

Also underrated I think is security by simplicity. OpenSSH should be extremely simple and easy to understand, such that every proposal and change could be easily scrutinized. Cryptographic constructions themselves are almost mathematically proven invulnerable, then a small codebase can go most of the way to mathematically provable security (bonus points for formal verification).

But for this kind of system there's usually some kind of human vulnerability (e.g. system updates for your distro) in the loop such that the community needs to remain watchful. (It's fun to consider an application that's proven correct and doesn't need updating every again, but usually that's not practical)

Re: Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

#218

Post by deepbreath »

deepbreath wrote: post #206
ulrikrasmussen wrote: post #94

Earlier quoted context omitted.

What if the knocking sequence was derived from a TOTP secret?

Don't have to do anything too complicated. Here's the knocker code in a short Bash script, produced by GPT4: ~ % gpt4 'write a very short bash script that takes the number stored in ~/.ssh/knock_seq, increments it by 1 and saves it to the file. It then takes the new number and concatenates it with the value stored in the file ~/.ssh/secret. It pipes the resulting string to sha1sum, spitting out binary. It then takes…

The knockee PoC should also be straightforward, can use socat + udp-listen + fork with a script that checks that input matches `sha1sum(secret||num)||num` and `num>previously_seen_num`, and if so, adds an iptables rule.

This should prevent against replays. Throw in some rate limits somewhere maybe to not get DDoSed, especially if you let socat `fork`.

Re: Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

#219

Post by bawolff »

MuffinFlavored wrote: post #182
acdha wrote: post #147

Earlier quoted context omitted.

That’s what the compromised build stage did. It’s really interesting to read if you want to see the details of how a sophisticated attacker works: https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78b... https://gynvael.coldwind.pl/?lang=en&id=782

> build-to-host.m4 I wasn't aware that the rogue maintainer was able to commit himself without any PR review (or he snuck it through PR review) rogue steps in the build process as well that went unnoticed so that he could bundle decompressed `xz` streams from test data, that also patched output .so files well enough to add hooking code to them. How many "process failures" are described in that process that exist in e…

> How many "process failures" are described in that process that exist in every OSS repo with volunteer unknown untrusted maintainers?

What process failures actually happened here? What changes in process do you think would have stopped this?

Re: Xzbot: Notes, honeypot, and exploit demo for the xz backdoor

#220

Post by slaymaker1907 »

pixl97 wrote: post #32
jobs_throwaway wrote: post #17

Imagine how frustrating it has to be for the attacker to meticulously plan and execute this and get foiled so late in the game, and so publicly

So who's started scanning open source lib test cases for more stuff like this?

I think it would help to try and secure signing infrastructure as much as possible. First of all, try to have 3 devs for any given project (this is the most difficult step). Make sure logging into or changing any of the signing stuff requires at least 2 of the devs, one to initiate the JIT and one to monitor.

Additionally, take supply chain steps like requiring independent approval for PRs and requiring the signing process to only work with automated builds. Don't allow for signing bits that were built on a dev machine.

Finally, I think it would help to start implementing signing executables and scripts which get checked at runtime. Windows obviously does executable signing, but it largely doesn't sign stuff like Python scripts. JS in the browser is kind of signed given that the whole site is signed via https. It's not perfect, but it would help in preventing one program from modifying another for very sensitive contexts like web servers.

Post reply on HN