Live data from Hacker News

Cloudflare 1.1.1.1 Incident on July 14, 2025

blog.cloudflare.com

151–160 of 391 posts

Re: Cloudflare 1.1.1.1 Incident on July 14, 2025

#151

An outage of roughly 1 hour is 0.13% of a month or 0.0114% of a year. It would be interesting to see the service level objective (SLO) that cloudflare internally has for this service. I've found https://www.cloudflare.com/r2-service-level-agreement/ but this seems to be for payed services, so this outage would put July in the " = 99.0%" bucket, so you'd get a 10% refund for the month if you payed for it.

Probably 99.9% or better annually just from a 'maintaining reputation for reliability' standpoint.

What really matters with these percentages is whether it’s per month or per year. 99.9% per year allows for much longer outages than 99.9% per month.

Re: Cloudflare 1.1.1.1 Incident on July 14, 2025

#152

Earlier quoted context omitted.

You are right infrastructure is important. But opposite to tap water there are a lot of different free DNS resolvers that can be used. And I don't see how my taxes funded CFs DNS service. But my ISP fee covers their DNS resolving setup. That's the reason why I wrote > a service that's free of charge Which CF is.

DNS shouldn't be privatized at all since it's a critical part of internet infrastructure, however at the same time the idea that somehow it's something a corporation should be allowed to sell to you at all (or "give you for free") is silly given that the service is meaningless without the infrastructure of the internet, which is built by governments (through taxes). I can't even think of an equivalent it's so ridicul…

DNS is not privatized. This is not about the root DNS servers, it's just about one of many free resolvers out there - in this case one of the bigger and popular ones.

Re: Cloudflare 1.1.1.1 Incident on July 14, 2025

#154
post #136
post #88

Earlier quoted context omitted.

I think normally you pair 1.1.1.1 with 1.0.0.1 and, if I understand this correctly, both were down.

8.8.8.8 + 1.1.1.1 is stable and mostly safe

Windows 11 does not allow using this combination

Re: Cloudflare 1.1.1.1 Incident on July 14, 2025

#155
post #88

Earlier quoted context omitted.

I think normally you pair 1.1.1.1 with 1.0.0.1 and, if I understand this correctly, both were down.

Just pair 1.1.1.1 with 9.9.9.9 (Quad9) so you have fault tolerance in terms of provider as well.

Windows 11 does not allow using this combination

Re: Cloudflare 1.1.1.1 Incident on July 14, 2025

#156

Earlier quoted context omitted.

In general, the idea of DNS's design is to use the DNS resolver closest to you, rather than the one run by the largest company. That said, it's a good idea to specifically pick multiple resolvers in different regions, on different backbones, using different providers, and not use an Anycast address, because Anycast can get a little weird. However, this can lead to hard-to-troubleshoot issues, because DNS doesn't alwa…

What’s your recommendation for finding the dns resolver closest to me? I currently use 1.1 and 8.8, but I’m absolutely open to alternatives.

Windows 11 doesn't allow using that combination

Re: Cloudflare 1.1.1.1 Incident on July 14, 2025

#157

Earlier quoted context omitted.

Its not wrong for smaller companies. But there's an argument that a big system critical company/provider like Cloudflare should be able to afford its own always on team with a night shift.

I think it is reasonable if the alarm trigger time is, say 5-10% of the time required to fix most problems.

Instead of downvoting me, I'd like to know why this is not reasonable?

Re: Cloudflare 1.1.1.1 Incident on July 14, 2025

#158
post #83

Earlier quoted context omitted.

On Android, in Settings, Network & internet, Private DNS, you can only provide one in "Private DNS provider hostname" (AFAIK). Btw, I really don't understand why it does not accept an IP (1.1.1.1), so you have to give an address (one.one.one.one). It would be more sensible to configure a DNS server from an IP rather than from an address to be resolved by a DNS server :/

Private DNS on Android refers to 'DNS over HTTPS' and would normally only accept a hostname. Normal DNS can normally be changed in your connection settings for a given connection on most flavours of Android.

No, it is not DNS over HTTPS it is DNS over TLS, which is different.

Re: Cloudflare 1.1.1.1 Incident on July 14, 2025

#159
post #71

> For many users, not being able to resolve names using the 1.1.1.1 Resolver meant that basically all Internet services were unavailable. Don't you normally have 2 DnS servers listed on any device. So was the second also down, if not why didn't it go to that.

Yup. I have Cloudfare and Quad9

Re: Cloudflare 1.1.1.1 Incident on July 14, 2025

#160

Earlier quoted context omitted.

There's a constant tension between speed of detection and false positive rates. Traditional monitoring systems like Nagios and Icinga have settings where they only open events/alerts if a check failed three times in a row, because spurious failed checks are quite common. If you spam your operators with lots of alerts for monitoring checks that fix themselves, you stress the unnecessarily and create alert blindness, b…

At Cloudflare’s scale on 1.1.1.1, I’d imagine you could do something comparatively simple like track ten-minute and ten-second rolling averages (I know, I know, I make that sound much easier and more practical than it actually would be), and if they differ by more than 50%, sound the alarm. (Maybe the exact numbers would need to be tweaked, e.g. 20 seconds or 80%, but it’s the idea.) Were it much less than 1.1.1.1 it…

I'm sure some engineer at cloudflare is evaluating something like this right now, and try it on historical data how many false positives that would've generated in the past, if any.

Thing is, it's probably still some engineering effort, and most orgs only really improve their monitoring after it turned out to be sub-optimal.

Post reply on HN