Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The xz sshd backdoor rabbithole goes quite a bit deeper

twitter.com

Post reply on HN
Print view View original

151–160 of 310 posts

Re: The xz sshd backdoor rabbithole goes quite a bit deeper

#151

Post by jjav »

timschmidt wrote: post #113
nick238 wrote: post #105

Earlier quoted context omitted.

I picture some division in [nation-state] where they're constantly creating personas, slowly working all sorts of languishing open source packages with few maintainers (this is the actual hard, very slow part), then once they have a bit of an in, they could recruit more technical expertise. The division is run by some evil genius who knows this could pay off big, but others are skeptical, so their resources are prett…

Moxie's reasons for disallowing Signal distribution via F-droid always rang a little flat to me ( https://github.com/signalapp/Signal-Android/issues/127 ). Lots of chatter about the supposedly superior security model of Google Play Store, and as a result fewer eyes independently building and testing the Signal code base. Everyone is entitled to their opinions, but independent and reproducible builds seem like a net p…

> supposedly superior security model of Google Play Store

Let's never forget that the google play store requires giving google the ability to modify your app code in any way they want before making it available for download. Oh sure, that backdoor will never be abused.

Re: The xz sshd backdoor rabbithole goes quite a bit deeper

#152

Post by intelVISA »

sureglymop wrote: post #96

Does anyone have a good explanation or introduction into the performance testing that was done to find this? And how to get started? Actually measuring performance always seemed to be a very hard task and I'd like to be able to do similar testing as the person which found this backdoor.

> measuring performance always seemed to be a very hard task

It's not hard: it's either easy, or impossible, depending on the culture at your shop.

At the basic level it's trivial - you instrument code and interpret the results against the HW and lower level machine counters.

Depending on $lang you have many good libraries for this kicking around, the hard part is working with a corp that values performance (99% do not) so none of the required mindset and surrounding infra will be setup to permit this in any useful capacity.

Re: The xz sshd backdoor rabbithole goes quite a bit deeper

#153

Post by dmitrygr »

Jasper_ wrote: post #132
orbital-decay wrote: post #119

Earlier quoted context omitted.

You probably have too high expectations when you hear the "state-sponsored" part. Every large organization will inevitably end up like any other. They also have bureaucracy, deadlines, production cycle, poor communication between teams, the recent iOS "maybe-a-backdoor" story also shows that they don't always care about burning the vulnerabilities because they amassed a huge pile of them.

Eh. Take a look at other state-sponsored attackers. We know they have 0-days for iOS, we know they've been used, but even Apple doesn't know what they are since they are so good at hiding their tracks. I don't think a state-sponsored attack would upload their payload to the git repo and tarball for all to stare at after it's been found out, which only took about a month. NSO Group built a turing-complete VM out of a…

> is clownshoes by comparison

The quality of work you attract in part depends on how much you pay. Go check out how much is paid for a persistent iOS exploit, compared to a Linux user space exploit. From that, you may draw conclusions about their relative perceived difficulty and desirability. This will explain why iOS exploits are done more professionally. They are rarer, much better paid, and thus attract a better audience and more work on guarding them from discovery.

Re: The xz sshd backdoor rabbithole goes quite a bit deeper

#154

Post by jwilk »

heyoni wrote: post #52
ufmace wrote: post #14

The weird thing about this one is how it seems super professional in some ways, and rather amateur in others. Professional in the sense of spending a long time building up an identity that seemed trustworthy enough to be made maintainer of an important package, of probably involving multiple people in social manipulation attacks, of not leaking the true identity and source of the attack, and the sophistication and ob…

I thought performance was actually fine? It only dragged when using valgrind, hence the rhetoric that it took some really unlikely circumstances for it to be detected that quickly.

No, it wasn't fine. From the advisory (https://news.ycombinator.com/item?id=39865810):

> With the backdoored liblzma installed, logins via ssh become a lot slower.

Re: The xz sshd backdoor rabbithole goes quite a bit deeper

#155

Post by cesarb »

rgmerk wrote: post #59
ufmace wrote: post #14

The weird thing about this one is how it seems super professional in some ways, and rather amateur in others. Professional in the sense of spending a long time building up an identity that seemed trustworthy enough to be made maintainer of an important package, of probably involving multiple people in social manipulation attacks, of not leaking the true identity and source of the attack, and the sophistication and ob…

Events can happen to anyone, even competent state-sponsored organisations. And intelligence agencies are sometimes rather less ruthlessly competent than imagined (Kremlin assisinations in the UK have been a comedy of errors [1]). Maybe another backdoor, or alternative access mechanism they were using, got closed and they wanted another one in a hurry. [1] https://en.wikipedia.org/wiki/Poisoning_of_Alexander_Litvine..…

> Maybe another backdoor, or alternative access mechanism they were using, got closed and they wanted another one in a hurry.

Or maybe the opportunity window for the mechanism this backdoor would use was closing. According to the timeline at https://research.swtch.com/xz-timeline there was a github comment from poettering at the end of January which implied that the relevant library would be changed soon to lazy load liblzma ("[...] Specifically the compression libs, i.e. libbz2, libz4 and suchlike at least. And also libpam. So expect more like this sooner or later."), as in fact happened a month later. The attacker had to get the backdoor into the targeted Linux distributions before the systemd change got into them.

Of course, the attacker could instead take the loss and abandon the approach, but since they had written that amount of complex code, it probably felt hard to throw it all away.

Re: The xz sshd backdoor rabbithole goes quite a bit deeper

#156

Post by cesarb »

rav wrote: post #109
sho_hn wrote: post #88

Earlier quoted context omitted.

This refers to the fact that systemd was planning to drop the dependency on liblzma (the conpression library installed by xz), and instead dlopen it at runtime when needed. Not for security reasons, but to avoid pulling the libs into initramfs images. The backdoor relies on sshd being patched to depend on libsystemd to call sd_notify(), which several distros had done. OpenSSH has since merged a new patch upstream tha…

> The backdoor relies on sshd being patched to depend on libsystemd to call sd_notify I remember when we added sd_notify support to our services at work, I was wondering why one would pull in libsystemd as a dependency for this. I mean, there's a pure-Python library [1] that basically boils down to: import os, socket def notify(state=b"READY=1"): sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) addr = os.geten…

> I was wondering why one would pull in libsystemd as a dependency for this. I mean, there's a pure-Python library [...] With proper error handling, that's about 50 lines of C code.

There's also a pure C library (libsystemd itself) which already does all that, and you don't need to test all the error handling cases in your 50 lines of C code. It makes sense to use the battle-tested code, instead of writing your own.

Re: The xz sshd backdoor rabbithole goes quite a bit deeper

#157

Post by Rygian »

renk wrote: post #150
ufmace wrote: post #14

The weird thing about this one is how it seems super professional in some ways, and rather amateur in others. Professional in the sense of spending a long time building up an identity that seemed trustworthy enough to be made maintainer of an important package, of probably involving multiple people in social manipulation attacks, of not leaking the true identity and source of the attack, and the sophistication and ob…

That performance regression could also be a way to identify compromised systems.

If all systems are compromised you don't need to identify anymore.

Re: The xz sshd backdoor rabbithole goes quite a bit deeper

#158

Post by XorNot »

asveikau wrote: post #125
nick238 wrote: post #105

Earlier quoted context omitted.

I picture some division in [nation-state] where they're constantly creating personas, slowly working all sorts of languishing open source packages with few maintainers (this is the actual hard, very slow part), then once they have a bit of an in, they could recruit more technical expertise. The division is run by some evil genius who knows this could pay off big, but others are skeptical, so their resources are prett…

I don't think we should assume a state actor. We don't know. It's kind of similar to stuxnet but attacking Linux distros is so broad and has such a huge risk of being exposed, as it was within a few weeks of deployment. A good nation state attack would put more effort into not being caught. But we don't know. So maybe I'm wrong.

Assuming a state-actor is a cope though. It's looking at the problem and saying "well we were fighting god himself, so really what could we have done?"

Whereas given the number of identities and time involved, the thing we really see is "it took what, 2-3 or burner email accounts and a few dozen hours over 2 years to almost hack the world?"

The entire exploit was within the scope of capability of one guy. Telling ourselves "nation-state" is pretending there isn't a really serious problem.

Re: The xz sshd backdoor rabbithole goes quite a bit deeper

#159

Post by ThePowerOfFuet »

orbital-decay wrote: post #136
Jasper_ wrote: post #132

Earlier quoted context omitted.

Eh. Take a look at other state-sponsored attackers. We know they have 0-days for iOS, we know they've been used, but even Apple doesn't know what they are since they are so good at hiding their tracks. I don't think a state-sponsored attack would upload their payload to the git repo and tarball for all to stare at after it's been found out, which only took about a month. NSO Group built a turing-complete VM out of a…

That's why I mentioned the recent story. [0] [1] "Apple doesn't know" when the chain uses an internal backdoor in Apple hardware is a... stretch. And the chain gives the strong vibes of corporate-style development, with all its redundancy and mismatch between two parts. It's not alchemy, really. [0] https://securelist.com/operation-triangulation-the-last-hard... [1] https://news.ycombinator.com/item?id=38783112 - HN…

Re the secret knock in the Apple silicon, a friend of mine once said "that's how you lose the NOBUS on a backdoor", and I think they were absolutely right.

The one thing which most leads me to believe this was an intentional backdoor? The S-boxes.

Re: The xz sshd backdoor rabbithole goes quite a bit deeper

#160

Post by XorNot »

cesarb wrote: post #156
rav wrote: post #109

Earlier quoted context omitted.

> The backdoor relies on sshd being patched to depend on libsystemd to call sd_notify I remember when we added sd_notify support to our services at work, I was wondering why one would pull in libsystemd as a dependency for this. I mean, there's a pure-Python library [1] that basically boils down to: import os, socket def notify(state=b"READY=1"): sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) addr = os.geten…

> I was wondering why one would pull in libsystemd as a dependency for this. I mean, there's a pure-Python library [...] With proper error handling, that's about 50 lines of C code. There's also a pure C library (libsystemd itself) which already does all that, and you don't need to test all the error handling cases in your 50 lines of C code. It makes sense to use the battle-tested code, instead of writing your own.

The problem is people keep focusing on the libsystemd element because systemd has it's big hate-on crew and the vector was for what's deemed "simple".

The better question though is...okay, what if the code involved was not simple? xz is a full compression algorithm, compressors have been exploit vectors for a while, so rolling your own is a terrifically bad idea in almost all cases. There's plenty of other more sophisticated libraries as well where you could've tried to pull the exact same trick - there's nothing about it being a "simple" inclusion in this case which implies vendoring or rolling your own is a good mitigation.

The saying goes that everyone is always preparing to fight the last war, not the next (particularly relevant because adversaries are likely scouring OSS looking for other projects that might be amenable to this sort of attack - how many applications have network access these days? An RCE doesn't need to be in sshd).

Post reply on HN