What's that about a hijack?
Related, non-causal event: BGP origin hijack of 1.1.1.0/24 exposed by withdrawal of routes from Cloudflare. This was not a cause of the service failure, but an unrelated issue that was suddenly visible as that prefix was withdrawn by Cloudflare.
Cloudflare 1.1.1.1 Incident on July 14, 2025
81–90 of 391 posts
Re: Cloudflare 1.1.1.1 Incident on July 14, 2025
#82Re: Cloudflare 1.1.1.1 Incident on July 14, 2025
#83> For many users, not being able to resolve names using the 1.1.1.1 Resolver meant that basically all Internet services were unavailable. Don't you normally have 2 DnS servers listed on any device. So was the second also down, if not why didn't it go to that.
Btw, I really don't understand why it does not accept an IP (1.1.1.1), so you have to give an address (one.one.one.one). It would be more sensible to configure a DNS server from an IP rather than from an address to be resolved by a DNS server :/
Re: Cloudflare 1.1.1.1 Incident on July 14, 2025
#841.1.1.1 does not operate in isolation. It is designed to be used in conjunction with 1.0.0.1. DNS has fault tolerance built in. Did 1.0.0.1 go down too? If so, why were they on the same infrastructure? This makes no sense to me. 8.8.8.8 also has 8.8.4.4. The whole point is that it can go down at any time and everything keeps working. Shouldn’t the fix be to ensure that these are served out of completely independent s…
Re: Cloudflare 1.1.1.1 Incident on July 14, 2025
#85Earlier quoted context omitted.
Are we meant to use a domain? I've always just used the IP.
You need a domain in order to get the s in https to work
LetEncrypt are trialling ip address https/TLS certificates right now:
https://letsencrypt.org/2025/07/01/issuing-our-first-ip-addr...
They say:
"In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale."
Re: Cloudflare 1.1.1.1 Incident on July 14, 2025
#86> For many users, not being able to resolve names using the 1.1.1.1 Resolver meant that basically all Internet services were unavailable. Don't you normally have 2 DnS servers listed on any device. So was the second also down, if not why didn't it go to that.
Re: Cloudflare 1.1.1.1 Incident on July 14, 2025
#87Earlier quoted context omitted.
Google is serving you ads, CF isn’t. And it’s not conspiracy theory - it was very suspicious when we did some testing on small, aware group. The traffic didn’t look like being handled anonymously at Google side
Unless the privacy policy changed recently, Google shouldn't be doing anything nefarious with 8.8.8.8 DNS queries.
Re: Cloudflare 1.1.1.1 Incident on July 14, 2025
#88> For many users, not being able to resolve names using the 1.1.1.1 Resolver meant that basically all Internet services were unavailable. Don't you normally have 2 DnS servers listed on any device. So was the second also down, if not why didn't it go to that.
Re: Cloudflare 1.1.1.1 Incident on July 14, 2025
#89Earlier quoted context omitted.
I disagree. The actual root cause here is shrouded in jargon that even experienced admins such as myself have to struggle to parse. It’s corporate newspeak. “legacy” isn’t a clear term, it’s used to abstract and obfuscate. > Legacy components do not leverage a gradual, staged deployment methodology. Cloudflare will deprecate these systems which enables modern progressive and health mediated deployment processes to pr…
If you carry on reading, its quite obvious they misconfigured a service and routed production traffic to that instead of the correct service, and the system used to do that was built in 2018 and is considered legacy (probably because you can easily deploy bad configs). Given that, I wouldn't say the summary is "inscrutable corporatese" whatever that is.
It's carefully written so my boss's boss thinks he understands it, and that we cannot possibly have that problem because we obviously don't have any "legacy components" because we are "modern and progressive".
It is, in my opinion, closer to "intentionally misleading corporatese".
Re: Cloudflare 1.1.1.1 Incident on July 14, 2025
#90Earlier quoted context omitted.
Related, non-causal event: BGP origin hijack of 1.1.1.0/24 exposed by withdrawal of routes from Cloudflare. This was not a cause of the service failure, but an unrelated issue that was suddenly visible as that prefix was withdrawn by Cloudflare.
So someone just started advertising the prefix when it was up for grabs? That’s pretty funny