Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DNSSEC disruption affecting .de domains – Resolved

status.denic.de

Post reply on HN
Print view View original

421–430 of 440 posts

Re: DNSSEC disruption affecting .de domains – Resolved

#421

Post by pocksuppet »

amluto wrote: post #405
phicoh wrote: post #339

Earlier quoted context omitted.

If DNSSEC is part of your security model, you want local validation. Not relying on third party resolver that you don't have a contract with. Beyond that, DNS has the AD bit. If you need DNSSEC secure data (for example for the TLSA record), then when Cloudflare turns off DNSSEC validation, the AD bit will be clear and things will stop working.

Am I the only one who thinks that the AD bit is about as useful as the RFC 3514 evil bit? We have this elaborate, complex, and extremely fragile cryptographic system behind DNSSEC and we distill it down to one single bit that we carry over unauthenticated links. Why? At least WebPKI answers the right question: should I trust a particular claim to represent host.domain at the time in the following range? (Of course it…

The intention is clearly that the client is a minimal implementation that will only forward a request to a resolver it trusts. The fact that Cloudflare and Google have convinced us all to use Cloudflare's and Google's resolvers is the problem.

DNSSEC and WebPKI both rely on chains of trust. If the problem was that .de's keys expired, you'd have the same problem when Let's Encrypt's keys expired.

Re: DNSSEC disruption affecting .de domains – Resolved

#422

Post by daneel_w »

Avamander wrote: post #222
daneel_w wrote: post #206

Earlier quoted context omitted.

Which is fine. Not because KSK rollover is supposedly complicated, but if you can't manage to keep your private keys and PKI safe in the first place then key rotation is just a security circus trick. But if you do know how to keep them safe, then...

It is not fine. Keeping key material safe is not a boolean between "permanently safe" and "leaks immediately". Keeping key material secure for more than a decade while it's in active use is vastly more complex than keeping it secure for a month, until it rotates. For all we know, some ex-employee might be walking around with that KSK, theoretically being able to use it for god knows what for an another decade.

Yeah, theoretically. They "only" need continued access to CF's internal systems. Surely you're aware that the ZSK is confined to your zone and can be rotated as much as you want without having to involve the root/registrar, and with none of the risks or consequences of not knowing how to perform a KSK rollover?

What's your take on the conundrum of Amazon Trust's 20+ year root cert, with which they sign a 5+ year intermediate, with which they sign a 2-month leaf?

Re: DNSSEC disruption affecting .de domains – Resolved

#423

Post by Bender »

tptacek wrote: post #418
Bender wrote: post #292

Earlier quoted context omitted.

That's cool, ty for that. The only one I put credentials into is Amazon it is unsigned. [1] There probably needs to be a DNSSECv2 .vbis that reduces risk somehow to get more adoption. [1] - https://dnssec-analyzer.verisignlabs.com/amazon.com

For what it's worth, technically we're already on something like DNSSEC-ter or DNSSEC-quater. -bis was back in the early 2000s with the typecode roll. It was really called DNSSEC-bis!

It was really called DNSSEC-bis!

That's too funny. I was just kidding. Back in the day Ericsson always added that to their upgraded product lines (Including GSM and what-not)

Re: DNSSEC disruption affecting .de domains – Resolved

#424

Post by Bender »

pocksuppet wrote: post #420
Bender wrote: post #292

Earlier quoted context omitted.

That's cool, ty for that. The only one I put credentials into is Amazon it is unsigned. [1] There probably needs to be a DNSSECv2 .vbis that reduces risk somehow to get more adoption. [1] - https://dnssec-analyzer.verisignlabs.com/amazon.com

Do we know what their root mistake was? I've studied and deployed DNSSEC, and as I see it, the current version is pretty much the simplest thing that could possibly work, given the way DNS works.

The root cause of the disruption has not yet been fully identified. DENIC’s technical teams are working intensively on analysis and on restoring stable operations as quickly as possible.

That's their current official statement. I could guess but I would rather wait until they have an official statement. I would imagine they must know but they are probably going back and forth with their legal team to word it very carefully, or at least that is what I would be doing if I were in their situation.

Re: DNSSEC disruption affecting .de domains – Resolved

#425

Post by throw0101c »

qazwsxedchac wrote: post #111
krystofbe wrote: post #7

Looks like a DNSSEC issue, not a nameserver outage. Validating resolvers SERVFAIL on every .de name with EDE: RRSIG with malformed signature found for a0d5d1p51kijsevll74k523htmq406bk.de/nsec3 (keytag=33834) dig +cd amazon.de @8.8.8.8 works, dig amazon.de @a.nic.de works. Zone data is intact, DENIC just published an RRSIG over an NSEC3 record that doesn't validate against ZSK 33834. Every validating resolver therefor…

So a single configuration mistake in a single place wiped out external reachability of a major economy. It happened in the evening local time and should be fixable, modulo cache TTLs, by morning. This will limit the blast radius somewhat. Still, at this level, brittle infrastructure is a political risk. The internet's famous "routing around damage" isn't quite working here. Should make for an interesting post mortem.

> So a single configuration mistake in a single place wiped out external reachability of a major economy.

No different than a bunch of BGP issues we've seen over the years.

And you don't even need DNSSEC for DNS to break things: reminder of the October 2025 AWS outage:

* https://www.akamai.com/blog/security/when-cloud-breaks-lesso...

Re: DNSSEC disruption affecting .de domains – Resolved

#426

Post by throw0101c »

ondohotola wrote: post #400
icedchai wrote: post #391

Earlier quoted context omitted.

It's simple enough to get a secondary DNS server somewhere and put it on $5/month VPS. I use BIND and DNS replication (AXFR/IXFR) handles it.

Have you ANY clue about the size of .DE's name server infrastructure?

> Have you ANY clue about the size of .DE's name server infrastructure?

Is it more or less than the F-root server run by ISC?

* https://www.isc.org/f-root/

If you want, you can even request your own instance (a 1U Dell):

* https://www.isc.org/froot-process/

Or an instance of ICANN's L-root server, also 1U:

* https://www.dns.icann.org/imrs/

Would .de have more, or less, traffic than some of the root servers?

Re: DNSSEC disruption affecting .de domains – Resolved

#427

Post by tptacek »

Bender wrote: post #423
tptacek wrote: post #418

Earlier quoted context omitted.

For what it's worth, technically we're already on something like DNSSEC-ter or DNSSEC-quater. -bis was back in the early 2000s with the typecode roll. It was really called DNSSEC-bis!

It was really called DNSSEC-bis! That's too funny. I was just kidding. Back in the day Ericsson always added that to their upgraded product lines (Including GSM and what-not)

Right, it's OSI/ITU-speak, and it's ironic to see it applied at IETF.

Re: DNSSEC disruption affecting .de domains – Resolved

#428

Post by akerl_ »

pocksuppet wrote: post #421
amluto wrote: post #405

Earlier quoted context omitted.

Am I the only one who thinks that the AD bit is about as useful as the RFC 3514 evil bit? We have this elaborate, complex, and extremely fragile cryptographic system behind DNSSEC and we distill it down to one single bit that we carry over unauthenticated links. Why? At least WebPKI answers the right question: should I trust a particular claim to represent host.domain at the time in the following range? (Of course it…

The intention is clearly that the client is a minimal implementation that will only forward a request to a resolver it trusts. The fact that Cloudflare and Google have convinced us all to use Cloudflare's and Google's resolvers is the problem. DNSSEC and WebPKI both rely on chains of trust. If the problem was that .de's keys expired, you'd have the same problem when Let's Encrypt's keys expired.

> If the problem was that .de's keys expired, you'd have the same problem when Let's Encrypt's keys expired.

Even this incident proves that’s not the case.

If LetsEncrypt has a temporary availability issue, my users don’t notice unless it spans longer than my need to renew a cert.

If LetsEncrypt has a CA cert expire, I can get a cert from another provider.

If DENIC’s DNSSEC records break, either due to an operational error or an expiry issue, my .de site becomes inaccessible and my users see a DNS lookup failure. My only option is to hope resolvers do what Cloudflare did, or move my site to a new TLD and just pray that TLD never has the same problem.

Re: DNSSEC disruption affecting .de domains – Resolved

#429

Post by weird-eye-issue »

liveoneggs wrote: post #385
weird-eye-issue wrote: post #284

Earlier quoted context omitted.

They didn't disable SSL you dingus.

it was an analogy to try highlighting how silly "security" is when it's opt-in and any intermediary can just disable it

I'm not sure you know how analogies work

Re: DNSSEC disruption affecting .de domains – Resolved

#430

Post by tptacek »

pocksuppet wrote: post #421
amluto wrote: post #405

Earlier quoted context omitted.

Am I the only one who thinks that the AD bit is about as useful as the RFC 3514 evil bit? We have this elaborate, complex, and extremely fragile cryptographic system behind DNSSEC and we distill it down to one single bit that we carry over unauthenticated links. Why? At least WebPKI answers the right question: should I trust a particular claim to represent host.domain at the time in the following range? (Of course it…

The intention is clearly that the client is a minimal implementation that will only forward a request to a resolver it trusts. The fact that Cloudflare and Google have convinced us all to use Cloudflare's and Google's resolvers is the problem. DNSSEC and WebPKI both rely on chains of trust. If the problem was that .de's keys expired, you'd have the same problem when Let's Encrypt's keys expired.

The WebPKI works end-to-end all the way to use devices; DNSSEC build an explicit client/server trust model into that. The former is obviously superior to the latter.

Yes, it's also quite damaging to DNSSEC's trust model that the world has transitioned to centralized resolver caches. But the fundamental problem we're talking about with the AD bit wouldn't vanish if 8.8.8.8 and 1.1.1.1 did too; instead, users would be even more reliant on ISP nameservers, which are literally the least trustworthy pieces of infrastructure on the entire Internet.

Post reply on HN