Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

bsky.app

Post reply on HN
Print view View original

811–820 of 862 posts

Re: XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

#811

Post by int_19h »

quotemstr wrote: post #239
jeroenhd wrote: post #12

Earlier quoted context omitted.

> However, since this is an RCE in the context of e.g. an sshd process itself, this means that sshd running as root would allow the payload to itself run as root. With the right sandboxing techniques, SELinux and mitigations could prevent the attacker from doing anything with root permissions. However, applying a sandbox to an SSH daemon effectively is very difficult.

Doesn't matter. This is a supply chain attack, not a vulnerability arising from a bug. All sandboxing the certificate parsing code would have done is make the author of the backdoor do a little bit more work to hijack the necessarily un-sandboxed supervisor process. Applying the usual exploit mitigations to supply chain attacks won't do much good. What will? Kill distribution tarballs. Make every binary bit for bit r…

... and stop adding random patches to upstream software, especially when we're talking about security-critical stuff that must absolutely not be released without a very thorough security review.

Re: XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

#812

Post by int_19h »

ynik wrote: post #686
Cloudef wrote: post #570

Earlier quoted context omitted.

Not true, it would be much harder to hook into openssl functions if the final executable was static [1], the only way is that if the openssl function this attack targeted, actually called a function from libxz. [1] https://sourceware.org/glibc/wiki/GNU_IFUNC Dynamic loading is relic of the past and cause of many headaches in linux ecosystem, in this case it also just obfuscates the execution path of the code more so…

This particular approach of hooking would be much harder; but a malicious xz has other options as well. It's already in the code path used by dpkg when unpacking packages for security updates, so it could just modify the sshd binary, or maybe add a rootkit to the next kernel security update. It seems foolish to change our systems to stop one of the steps the attacker used after their code was already running as root;…

There are other reasons to change, though. The main thing to consider here is that static linking is the "OG" way of doing things, and also the simplest and the most easily understandable one. There are also obvious perf benefits to it when it comes to optimizing compilers.

On the other hand, dynamic linking was originally more or less just a hack to deal with memory-restricted environments in the face of growing amounts of code. It was necessary at the time because we simply wouldn't have things like X or Windows without it way back when.

But RAM is nowhere near as sparse these days, and it could be even less so if there was a concerted push on hardware vendors to stop skimping on it. So why don't we remove the hack and get back to a simple model that is much easier to understand, implement, and audit?

Re: XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

#813

Post by int_19h »

saagarjha wrote: post #520
josephg wrote: post #477

Earlier quoted context omitted.

This was the backdoor we found. We found the backdoor with performance issues. Whats more likely - that this is the only backdoor like this in linux, or that there are more out there and this is the one we happened to find? I really hope someone is out there testing for all of this stuff in linux: - Look for system() calls in compiled binaries and check all of them - Look for uses of IFUNC - specifically when a libra…

All of this was obfuscated. None of this will be detectable with current static analysis techniques.

We have governments, which even in the face of budget crises and such tend to allocate enormous sums for "national security". Why not have them actually do something useful with that for once and do a manual line-by-line audit of all security-critical code that is underpinning our infrastructure?

Re: XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

#814

Post by int_19h »

denton-scratch wrote: post #654
pxc wrote: post #514

Earlier quoted context omitted.

It's not actually unusual for three-letter US agencies to be at odds with one another. But one possible reason is if the FBI is convinced that something the NSA is doing is illegal. They may not always be inclined to tolerate that.

> It's not actually unusual for three-letter US agencies to be at odds with one another. I'd noticed that; this seems to have been the case for a long time. You'd think that having state security agencies at war with one-another would be a disaster, but perhaps it's a feature: a sort of social "layered security". At any rate, it seems much better than having a bunch of state security agencies that all sing from the s…

It's a bog standard practice, actually, even if you look very far back to the ancient world. Having a single agency responsible for security of yourself and what you own is a bad idea because no matter how much you try ensure the loyalty of people in it, it's prone to, at the minimum, suppressing its own failures and magnifying its successes to make itself look better than it actually is, giving you a false sense of security. It is also the natural point from which to orchestrate a coup, which is something that can be used by your adversaries, but even without their involvement people working there eventually realize that they hold all the keys to the kingdom and there's little risk in them just taking over.

So rulers in all ages tended to create multiple different security apparatuses for themselves and their states, and often actively encouraged rivalries between them, even if that makes them less efficient.

Re: XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

#815

Post by josephg »

saagarjha wrote: post #791
josephg wrote: post #566

Earlier quoted context omitted.

IFUNC and landlock could be debugged pretty easily at runtime, just by adding some instrumentation.

Yes but that requires you to know that someone will use it beforehand.

Now that we know IFUNC can me misused like this, it would be pretty silly if we allow other, future exploits to use the same trick.

Re: XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

#816

Post by [deleted] »

ajross wrote: post #172
admax88qqq wrote: post #161

Earlier quoted context omitted.

How can sshd spawn interactive sessions for other users if it's sandboxed?

Plausibly by having set-user-ID capability but not others an attacker might need. But in the more common case it just doesn't: you have an sshd running on a dedicated port for the sole purpose of running some service or another under a specific sandboxed UID. That's basically the github business model, for example.

[deleted]

Re: XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

#817

Post by eto-san »

saagarjha wrote: post #496
CanaryLayout wrote: post #436

Earlier quoted context omitted.

Exactly. The attack came in by hitching a ride on to systemd. sshd is not the problem. the ldd/monolith architecture surrounding systemd is. What if I duplicated this attack but instead targeted dbus or any other thing that systemd is managing?

No, the problem is that someone had access to backdoor code that runs in a privileged process.

Tell us all, please, how the starting vector of this attack would affect statically compiled dropbear binary even with systemd's libsystemd pwnage? I am very cruious about your reasoning.

The fact, that the whole reason this library is even being pulled into the sshd daemon process, is some stupid stuff like readiness notification, which itself is utterly broken on systemd, by design (and thus is forever unfixable), and makes this even more tragic.

Don't put your head into the sand, just because of the controversial nature of the topic. Systemd was VERY accommodating in this whole fiasco.

Saddest part of all this is, that we know how to to do better. At least since Bernstein, OpenBSD and supervision community (runit/s6) guys solved it. Yet somehow we see same mistakes repeated again and again.

If you want to read about notification shenanigans and systemd's dubious (at best) decisions and implementation quality, read here: https://jdebp.uk/FGA/unix-daemon-readiness-protocol-problems...

I believe this is how you do readiness notification properly: http://skarnet.org/software/s6/notifywhenup.html

I.e. you fork and run little helper to write, or directly write a single byte(!), to notify supervisor over supervisor provided fd. It allows you to even privseparate your notifier stuff or do all the cute SELinux magic you need.

But that would be too simple, I guess, so instead we link like 10 completely unrelated libraries into sshd, liblzma being one of them, one of the most crucial processes on the machine. To notify supervisor that it's ready. Sounds about right, linux distros (and very specific ones at that).

Sshd should be sacred, nothing more than libc and some base cryptolibs (I don't remember whether it still needs ssl even) it needs.

Another great spot to break sshd is PAM, which has no place doing there either. Unfortunately it's hard dep. on most linux distros.

Maybe sshd should adopt kernel taint approach: as soon as any weird libraries (ie everything not libc and cryptolibs) are detected in sshd proces it should consider itself tainted. Maybe even seppuku itself.

The exploit could be, probably, somehow doable without systemd. But it would be much, much harder though.

Don't try to obfuscate that very fact from the discussion.

Re: XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

#818

Post by eto-san »

kbolino wrote: post #710
kimixa wrote: post #613

Earlier quoted context omitted.

Really what they're proposing here is a non-modifiable system, where the root is read-only and no user can modify anything important. Which is nice and all, but that implies a "parent" system that creates and deploys those systems. Which people likely want remote access to.. Probably by sshd...

You don't have to have an immutable system. You can limit the exposure of the system from RCE in sshd with SELinux without preventing legitimate users from administering the system. Granted that SELinux is overly complicated and has some questionable design decisions from a usability standpoint but it's not as limited or inflexible as many seem to think. It really can stop a system service running as "root" from doin…

Exactly! Very well done explaning!

Most people turn SELinux off anyway, so they have no clue how it operates.

DACs (discretionary, unix perms) are DACs and MACs (mandatory, SELinux) are MACs. They are mandatory - it's in their name.

Think of SELinux as completely orthogonal access control system, that can overturn any DAC decision, which it in fact does. SELinux language is much more featured than DAC language, it can express domain transitions.

Nobody here has inspected the sshd_t policies but I believe exec transition should be forbidden for arbitrary binaries (I hope).

That should in essence thwart arbitrary exec from remote key payload. If actual shellcode would be sent though (e.g. doing filesystem open/write/close), that is a little bit different.

Re: XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

#819

Post by eto-san »

eklitzke wrote: post #795
CanaryLayout wrote: post #456

Earlier quoted context omitted.

even easier is to STOP HOSTING SSHD ON IPV4 ON CLEARNET at minimum, ipv6 only if you absolutely must do it (it absolutely cuts the scans way down) better is to only host it on vpn even better is to only activate it with a portknocker, over vpn even better-better is to set up a private ipv6 peer-to-peer cloud and socat/relay to the private ipv6 network (yggdrasil comes to mind, but there's other solutions to darknet)…

This is a joke right? If you have password authentication disabled then it shouldn't matter how many thousands of times a day people are scanning and probing sshd. Port knockers, fail2ban, and things of that nature are just security by obscurity that don't materially increase your security posture. If sshd is written correctly and securely it doesn't matter if people are trying to probe your system, if it's not writt…

But ssh is written correctly. Now that other thing isn't. :D

I fail to see a problem here.

Re: XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

#820

Post by wepple »

saagarjha wrote: post #507
tux3 wrote: post #187

Earlier quoted context omitted.

I broadly agree, but I think we can draw a parallel with the arms race of new exploit techniques versus exploit protection. People still manage to write exploits today, but now you must find an ASLR leak, you must chain enough primitives to work around multiple layers of protection, it's generally a huge pain to write exploits compared to the 90s. Today the dynamic detection that we have for Linux packages seems thin…

This is an arms race that is largely won by attackers, actually. Sophisticated attacks are caught by them sometimes but usually the author has far more knowledge or cleverer tricks than the person implementing the checks, who is limited by their imagination of what they think an attacker might do.

I’m not sure that attackers “win” this arms race; the cost has gone through the roof and their chance of detection constantly increases.

Indeed, defenders are often outgunned, but at a minimum we’ve pushed the level of effort way into nation-state territory, which is a good start.

Post reply on HN