Live data from Hacker News

Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

devever.net

71–80 of 150 posts

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#71

The summary of the attack from https://notes.valdikss.org.ru/jabber.ru-mitm/ is very interesting: * The attacker managed to issue multiple SSL/TLS certificates via Let’s Encrypt for jabber.ru and xmpp.ru domains since 18 Apr 2023 * The Man-in-the-Middle attack for jabber.ru/xmpp.ru client XMPP traffic decryption confirmed to be in place since at least 21 July 2023 for up to 19 Oct 2023, possibly (not confirmed) since…

Is it given that this attack was done from within Hetzner?

As I understand the techniques applied; this could have also been done at another place on the network route to the targeted server.

Ie. by the telecommunications company delivering traffic into Hetzner.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#72
post #64
post #49

A quick warning on hetzner. I needed a personal bare metal machine so signed up. I was travelling and on an IP in a distant land so their sign up asked for secondary verification via PayPal. All passed and now it’s should get a server? Nope - next day their support emailed telling me they would not approve my account without… no word of a lie here… either 1: a fax of my passport info page or 2: a scan and email conta…

They had the option to send it encrypted with PGP. But yes, this reminds me of communist countries where you had to leave your ID at the hotel upon check in. The Stasi mentality lingers on and accomplishes nothing.

You don't seem to know anything about VPS administration.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#73
post #16

Great callout: > Don't use Cloudflare or similar services. See my article here for an explanation on why. If you use a service like this, you're basically already MitMing yourself. I wish more people would realize that when arguing on the internet about CAA, DNSSEC, NSA, etc. that none of it really matters. We willingly allow a government aligned entity to unwrap 20% of all TLS connections on the internet and peak in…

Cloudflare is horrible for privacy. It is also a bit of a sovereignty issue for European countries to have all their citizens web habits to be MITM by a forging power (no matter how friendly they seam). Edit: not even going in to the sovereignty issue of having an American private company effectively decide your internet regulations.

Cloudflare exists out of necessity for the most part. The alternatives to shield from large scale DDoS are all US American too.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#74
post #64
post #49

A quick warning on hetzner. I needed a personal bare metal machine so signed up. I was travelling and on an IP in a distant land so their sign up asked for secondary verification via PayPal. All passed and now it’s should get a server? Nope - next day their support emailed telling me they would not approve my account without… no word of a lie here… either 1: a fax of my passport info page or 2: a scan and email conta…

They had the option to send it encrypted with PGP. But yes, this reminds me of communist countries where you had to leave your ID at the hotel upon check in. The Stasi mentality lingers on and accomplishes nothing.

I think it is less Stasi and more so 30 euro dedicated servers with unmetered gigabit lines are ripe for bad clients.

You've got the general issue of abuse and fraud that all providers face. But I think there are two issues that make it worse for companies like Hetzner, OVH, and other low-cost providers:

1. Chargebacks are a big deal, both in terms of being cut off from payment networks but also the fees imposed, which are especially harsh if your margins are likely razor thin sometimes; looking at the server auctions right now, it's kind of wild that Hetzner manages to give you a place in a datacenter, 3700X, 64GB RAM, 2x1TB enterprise SSDs, and unmetered gigabit on a decent network for 33 euros while still making a profit.

2. I would imagine that attempting the proper credit-card theft kind of fraud is also more of an issue for low-cost providers, not only because of #1, but because I think you'd manage to keep and abuse servers bought with stolen money for a lot longer; I think legitimate owners of said cards are less likely to notice 30 euro charges every month compared to being robbed blind by unexpected AWS fees.

I've had to deal with anti-fraud paranoia from OVH, BuyVM, Hetzner, and many others, likely all for the same reasons as Hetzner.

Both Hetzner and OVH refused to provide me service without photo ID or a passport. BuyVM refused one of my Jordanian friends entirely unless he paid in crypto. And while minor in comparison, I've had to change my PayPal email to match my account email on BuyVM despite it literally previously being paypal@myfirstandlastname.tld.

Not meant to be a dig at BuyVM, btw, even though the crypto bit may seem harsh. I really like them; freedom to host pretty much anything that isn't straight-up illegal, even Tor exit nodes, the support is good, they're often around and transparent in the community chat, and it's hard to beat free BGP announces, up to 10Gbps speeds, anycasted VMs across Europe and the US for They provide great service, even if only to use as a DDoS-mitigated tunnel for more powerful servers elsewhere, or as a CDN, etc.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#75

The summary of the attack from https://notes.valdikss.org.ru/jabber.ru-mitm/ is very interesting: * The attacker managed to issue multiple SSL/TLS certificates via Let’s Encrypt for jabber.ru and xmpp.ru domains since 18 Apr 2023 * The Man-in-the-Middle attack for jabber.ru/xmpp.ru client XMPP traffic decryption confirmed to be in place since at least 21 July 2023 for up to 19 Oct 2023, possibly (not confirmed) since…

Is it given that this attack was done from within Hetzner? As I understand the techniques applied; this could have also been done at another place on the network route to the targeted server. Ie. by the telecommunications company delivering traffic into Hetzner.

Tele communication companies have internal “police groups” (where I am from - I expect it to be the same in Germany) that does nothing but service wiretapping requests from the police. Telcos are required by law to do this.

Them expanding into mtm https wiretapping is a new for me but maybe to be expected …

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#76

Earlier quoted context omitted.

> Both Hetzner and Linode network appear to be reconfigured specifically for this kind of attack for the XMPP service IP addresses This suggests a compromise of Hetzner and Linode network management.

> This suggests a compromise of Hetzner and Linode network management. No it doesn't. It suggests that they both complied with a request from law enforcement.

So… compromised.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#77

The summary of the attack from https://notes.valdikss.org.ru/jabber.ru-mitm/ is very interesting: * The attacker managed to issue multiple SSL/TLS certificates via Let’s Encrypt for jabber.ru and xmpp.ru domains since 18 Apr 2023 * The Man-in-the-Middle attack for jabber.ru/xmpp.ru client XMPP traffic decryption confirmed to be in place since at least 21 July 2023 for up to 19 Oct 2023, possibly (not confirmed) since…

XMPP supports channel binding, which is not mentioned in this post but would have prevented this attack. Unfortunately jabber.ru is running server software from 2016, and that old version doesn't support it.

Pinning is problematic these days, because certificates are short-lived and renewed frequently. Users would be constantly asked to accept new certificates on a monthly basis, and they wouldn't think twice about clicking 'Accept' on an attacker's certificate then.

Channel binding works regardless of the certificate, and ensures the TLS stream is terminated by the entity you exepect.

The focus on (lack of) CT/SCT support in XMPP clients puzzles me, because it would not have detected this attack at all, which used valid CT-logged certs from Let's Encrypt. SCT verification would help detect theoretical issuance from another CA if they had a strict CAA record in place (they did not). But even then, channel binding is more privacy-friendly and does not depend on a third party.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#78
post #44

Earlier quoted context omitted.

>affected 100% of the connections to XMPP STARTTLS port 5222 (not 5223) Why did they only target the STARTTLS port? On a related note, I would never use the STARTTLS port (opportunistic encryption) if I knew that the server had a regular TLS port...

Easier to conceal the attack. The MiTM attacker can pass through a command stream without STARTTLS. If they intercepted 5223 they would have to do their own client-side TLS handshake with the attacked server, which would look really obvious to anybody doing TLS fingerprinting on the server: all of a sudden, 100% of their clients have the exact same TLS fingerprint. Stop outsourcing your PKI to ICANN, folks. Domains a…

They were doing their own TLS handshake - that's how the attack was discovered (the attacker presented a different certicate, which eventually expired, presumably due to negligence). They were decrypting and re-encrypting.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#79

Earlier quoted context omitted.

Cloudflare is horrible for privacy. It is also a bit of a sovereignty issue for European countries to have all their citizens web habits to be MITM by a forging power (no matter how friendly they seam). Edit: not even going in to the sovereignty issue of having an American private company effectively decide your internet regulations.

Cloudflare exists out of necessity for the most part. The alternatives to shield from large scale DDoS are all US American too.

Which lets be honest isn't a problem that 99% of the sites using Cloudflare need to solve. Nobody is going to waste energy and time to attack your blog with your vacation photos.

The huge wave of DDoS extortion attacks that happened 3-4 years ago was mostly enabled by "booter" services that themselves hid from law enforcement behind Cloudflare.

Here is an example: https://therecord.media/feds-seize-ddos-booter-sites-in-late...

Feel free to punch any of the sites they mention into dns.coffee and look at the historical nameservers. All Cloudflare.

Re: Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

#80
post #53

Earlier quoted context omitted.

All the people working in the datacenter have that level of physical access. Unless they are very closely supervised they can do a lot of damage without anybody being the wiser until they get caught. I've been in (nominally very secure) DCs on behalf of customers and I've seen: - unlocked racks - doors open - temporary network cables and keyboards, monitors and mice attached to running systems - systems logged in lef…

Agree re: everything you said but wanted to add datadentre security staff are some of the most interesting characters I’ve encountered. Not sure I sleep as well at night after seeing what I saw.

Do tell, please; stories about "interesting characters" are often the best.
Post reply on HN