Live data from Hacker News

Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

devever.net

41–50 of 150 posts

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#41

The summary of the attack from https://notes.valdikss.org.ru/jabber.ru-mitm/ is very interesting: * The attacker managed to issue multiple SSL/TLS certificates via Let’s Encrypt for jabber.ru and xmpp.ru domains since 18 Apr 2023 * The Man-in-the-Middle attack for jabber.ru/xmpp.ru client XMPP traffic decryption confirmed to be in place since at least 21 July 2023 for up to 19 Oct 2023, possibly (not confirmed) since…

> * The attacker failed to reissue TLS certificate and MiTM proxy started to serve expired certificate on port 5222 for jabber.ru domain (Hetzner)

This is gold.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#42
post #29

Earlier quoted context omitted.

Cloudflare is horrible for privacy. It is also a bit of a sovereignty issue for European countries to have all their citizens web habits to be MITM by a forging power (no matter how friendly they seam). Edit: not even going in to the sovereignty issue of having an American private company effectively decide your internet regulations.

The EU regularly de-facto tries to decide regulations for other countries. All is fair in a globally connected world.

Yeah, but it generally goes in the direction of more privacy, environmental and labor standards. Less so from other countries.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#43
post #41

The summary of the attack from https://notes.valdikss.org.ru/jabber.ru-mitm/ is very interesting: * The attacker managed to issue multiple SSL/TLS certificates via Let’s Encrypt for jabber.ru and xmpp.ru domains since 18 Apr 2023 * The Man-in-the-Middle attack for jabber.ru/xmpp.ru client XMPP traffic decryption confirmed to be in place since at least 21 July 2023 for up to 19 Oct 2023, possibly (not confirmed) since…

> * The attacker failed to reissue TLS certificate and MiTM proxy started to serve expired certificate on port 5222 for jabber.ru domain (Hetzner) This is gold.

The absolute lack of giving a shit is one of your major clues this was a lawful intercept scenario.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#44

The summary of the attack from https://notes.valdikss.org.ru/jabber.ru-mitm/ is very interesting: * The attacker managed to issue multiple SSL/TLS certificates via Let’s Encrypt for jabber.ru and xmpp.ru domains since 18 Apr 2023 * The Man-in-the-Middle attack for jabber.ru/xmpp.ru client XMPP traffic decryption confirmed to be in place since at least 21 July 2023 for up to 19 Oct 2023, possibly (not confirmed) since…

>affected 100% of the connections to XMPP STARTTLS port 5222 (not 5223)

Why did they only target the STARTTLS port? On a related note, I would never use the STARTTLS port (opportunistic encryption) if I knew that the server had a regular TLS port...

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#45
post #8

The provider has access to the host, they can just inspect the job from the outside and you won’t be able to tell

The Hetzner one is a physical server. You would need to stage a "power outage" and backdoor it, which is probably not that easy - e.g. planting a kernel module which survives kernel upgrades and is pretty advanced at hiding itself (the article talks about analyzing raw memory dump).

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#47

The summary of the attack from https://notes.valdikss.org.ru/jabber.ru-mitm/ is very interesting: * The attacker managed to issue multiple SSL/TLS certificates via Let’s Encrypt for jabber.ru and xmpp.ru domains since 18 Apr 2023 * The Man-in-the-Middle attack for jabber.ru/xmpp.ru client XMPP traffic decryption confirmed to be in place since at least 21 July 2023 for up to 19 Oct 2023, possibly (not confirmed) since…

> Both Hetzner and Linode network appear to be reconfigured specifically for this kind of attack for the XMPP service IP addresses This suggests a compromise of Hetzner and Linode network management.

> This suggests a compromise of Hetzner and Linode network management.

No it doesn't. It suggests that they both complied with a request from law enforcement.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#48

> What would a perfect attacker do? If you had physical access to the computer, some sort of bus interception to exfiltrate data from the machine.

extremely difficult to get physical access in a datacenter

Assuming this was done at the government's request, I assume Hetzner is more than willing to comply with a court order mandating they allow them to monitor and physically size a machine.

And outside of nation-state requests, even ignoring the fact that someone could probably pay-off an employee, I think ease would depend a lot on the datacenter and target; judging by the awesome and hilarious story behind the Fremont Cabal accidentally becoming an internet exchange by essentially having some dude barely secretly slipping unauthorized cables into the raceways [1], I figure there are a lot of places where if your target is simply renting a couple rack units or single rack rather than an entire locked cage, you can probably get physical access by doing the same.

Also, a lot of Deviant Ollam's stories about industrial security and the dozens of ways he's broken into utility companies, server rooms, etc — mostly just by being confident, looking the part, bad doors [2], and badge cloning [3] — don't give me a ton of confidence that someone with skill couldn't feasibly either get direct access to servers they shouldn't, or at the very least, access to an important part of the supply chain for their target.

And speaking of supply chain, my processor died recently, so I ordered a brand-new in box replacement Ryzen, and when it arrived last night, out of curiosity, I wanted to see if I could get the CPU out of the box without breaking the tamper-evident authenticity seal...

... and about fifteen minutes later, after borrowing a syringe and hypodermic needle from my mom, a little bit of isopropyl alcohol, a blade from my safety razor, and a quick look at a video from LockPickingLawyer [4] and a couple from datagram at DEFCON's Tamper-Evident Village [5][6], I had the CPU out, put my old one for now, and re-applied the sticker with no visible damage to the box or seal.

All I had to do was tip it upside down at about 90-degrees, douse a little bit of the alcohol under the top of the seal, let gravity do most of the work, and then carefully lift the seal with the razor. After that, I just lightly squeezed the box to make the front tab come as forward as possible, and then carefully pushed the ear flaps down to prevent tearing, and then I was in.

I've seen others demonstrate it on older AMD boxes that had flexible cardboard in-place of the cooler, allowing them to pull the cardboard to make enough room for tools to get out the CPU without even touching the seal [7]. But in my case, it was a newer box with hard plastic inside where the cooler would've been, so that's why I went for the seal instead.

No surprise to me now that counterfeiting is rampant on Amazon, with people returning the box after putting in either random junk, dead Athlons covered by a counterfeit serial-matching IHS, or the cheapest socket-compatible CPU after deluding both and swapping the IHS.

I figure with a bit of practice and better tools, like Teflon spudgers and syringes, it'd be significantly easier to get past 99% of tamper-resistant/tamper-evident seals and into boxes you're not supposed to be without risking damage, and then you can intercept a package, compromise something critical, like the server BMC or firmware, reseal everything, and be on your way.

And given the relatively recent scare with loads of servers, including Dell and others, being shipped with "AMERICAN MEGATRANDS" labels on their BMC boards, with no one noticing until a YouTube commenter pointed it out during a teardown by ServeTheHome, I think it's totally feasible for an enemy to just compromise the entire physical supply-chain of a company, datacenter, or whatever else [8].

[1]: Oxide's On the Metal: Kenneth Finnegan - https://oxide.computer/podcasts/on-the-metal/kenneth-finnega...

[2]: Deviant Ollam @ Shakacon: The Search for the Perfect Door - https://www.youtube.com/watch?v=4YYvBLAF4T8

[3]: Deviant Ollam / Modern Rogue: Getting an RFID Implant - https://youtu.be/SZiRISGdQ4g?t=277

[4]: LockPickingLawyer: Did I Cheat On This Challenge? (Tamper-Sealed Abus) - https://www.youtube.com/watch?v=xUJtqvYDnkg

[5]: DEFCON 19: Introduction to Tamper Evident Devices - https://www.youtube.com/watch?v=W07ZpEv9Sog

[6]: DEFCON 30: Tamper Evident Village - https://www.youtube.com/watch?v=slhdowWjSuU

[7]: cycurious: How Counterfeiters replace CPU in Sealed Retail Box - https://www.youtube.com/watch?v=Bni8bgGlXDE

[8]: ServeTheHome: Dude this should NOT be in a Dell Switch… or HPE Supercomputer - https://www.servethehome.com/dude-dell-hpe-ami-american-mega...

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#49
A quick warning on hetzner. I needed a personal bare metal machine so signed up.

I was travelling and on an IP in a distant land so their sign up asked for secondary verification via PayPal. All passed and now it’s should get a server?

Nope - next day their support emailed telling me they would not approve my account without… no word of a lie here… either 1: a fax of my passport info page or 2: a scan and email containing the same.

I refused reminding them of GDPR and that email is at best opportunistically encrypted and at worst open to interception.

They replied stating they believed they were GDPR compliant because all they do is use the passport to verify the account and delete the document. They also said I could hide anything sensitive other than my name and date of birth!!

I suggested the process is not GDPR compliant as anyone could intercept unencrypted emails and that they should talk to a lawyer if they did not believe my assertion.

Within a short time the server was approved and online. I queried if they would revise their process in light of our interaction. They did not address the question.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#50
post #44

The summary of the attack from https://notes.valdikss.org.ru/jabber.ru-mitm/ is very interesting: * The attacker managed to issue multiple SSL/TLS certificates via Let’s Encrypt for jabber.ru and xmpp.ru domains since 18 Apr 2023 * The Man-in-the-Middle attack for jabber.ru/xmpp.ru client XMPP traffic decryption confirmed to be in place since at least 21 July 2023 for up to 19 Oct 2023, possibly (not confirmed) since…

>affected 100% of the connections to XMPP STARTTLS port 5222 (not 5223) Why did they only target the STARTTLS port? On a related note, I would never use the STARTTLS port (opportunistic encryption) if I knew that the server had a regular TLS port...

Easier to conceal the attack.

The MiTM attacker can pass through a command stream without STARTTLS. If they intercepted 5223 they would have to do their own client-side TLS handshake with the attacked server, which would look really obvious to anybody doing TLS fingerprinting on the server: all of a sudden, 100% of their clients have the exact same TLS fingerprint.

Stop outsourcing your PKI to ICANN, folks. Domains are not public keys.

Post reply on HN