Live data from Hacker News

Akamai Edge DNS was down

edgedns.status.akamai.com

211–220 of 229 posts

Re: Akamai Edge DNS was down

#211
post #148

Serious question, has anyone properly solved the issue of DNS as a single point of failure?

You can still hardcode IP addresses. Not sure most people realize DNS isn't actually needed, you know, except for convenience and all that.

Meh. Without DNS, or something similar, there really is no internet.

Obviously you are technically correct.

Re: Akamai Edge DNS was down

#212

Absolutely amazing how many billion $+ companies are single homed for DNS. I wonder how much they spend on multi-AZ redundant architectures...

Problem is, if your on Akamai’s CDN, only Akamai know where the local caches are. You need to be on their DNS only.

Re: Akamai Edge DNS was down

#213
post #161

Earlier quoted context omitted.

> Whatever multi-home means, why can't there just be one service provider that does that? Many of these outages are due to pushing broken artifacts or configuration to production. A single provider can pretty easily offer geographic or network topological redundancy, but administrative and/or technological independence is pretty hard to achieve in a single company.

I mean, I guess what I'm saying is that in theory a single provider could purposely keep two different departments that manage their own artifacts independently.

Records have to be kept in sync.

If one dept deletes a record and the other doesn’t, how do you decide who’s right?

You could add a third dept that gives them both orders, but now that third dept is a single point of failure.

Re: Akamai Edge DNS was down

#214

People don't believe me when I say how much DNS matters. So I wrote a song about it. https://soundcloud.com/ryan-flowers-916961339/dns-to-the-tun...

Sounds amazing! Do you maybe have a direct link? Soundcloud doesn't want us privacy-conscious users browsing their website :(

As you wish :) https://meetryanflowers.com/wp-content/uploads/2021/07/DNS.m...

Re: Akamai Edge DNS was down

#215
post #88

Earlier quoted context omitted.

because the way downdetector works is it just basically counts how many people are searching/visiting for down and if it's much higher than typical it flags the site as down. So if everyone searched "is google down" and visited the link on downdetector that was returned in the search, that would add to the downdetector count for that site. Downdetector doesn't actually know if the site is up or down.

I found this hard to believe, but it's correct. Downdetector only reports an issue if a significant number of users are impacted. To that end, Downdetector calculates a baseline volume of typical problem reports for each service monitored, based on the average number of reports for that given time of day over the last year. Downdetector’s incident detection system compares the current number of problem reports to thi…

What’s hard to believe? Downdetectors well known for being almost, but not quite, useless.

Probably reported Google as “down” because a whole bunch of people use the word “Google” when they mean “internet”.

Re: Akamai Edge DNS was down

#216
post #102

Earlier quoted context omitted.

because the way downdetector works is it just basically counts how many people are searching/visiting for down and if it's much higher than typical it flags the site as down. So if everyone searched "is google down" and visited the link on downdetector that was returned in the search, that would add to the downdetector count for that site. Downdetector doesn't actually know if the site is up or down.

So how do they reset status? The number of queries going down signifies return to normal status?

Yep

Re: Akamai Edge DNS was down

#217

I ran DNS servers, among other things, in the late 90s with better uptime than these "multi-DC/AZ/geo redundant" services everyone uses these days.

With all due respect, having also run auth DNS servers in the 90s, and seen the inside of Akamai’s CDN/DNS setup more recently, it isn’t remotely at the same level of scale or sophistication.

Re: Akamai Edge DNS was down

#218

Earlier quoted context omitted.

Sounds amazing! Do you maybe have a direct link? Soundcloud doesn't want us privacy-conscious users browsing their website :(

As you wish :) https://meetryanflowers.com/wp-content/uploads/2021/07/DNS.m...

I had a good laugh, thank you very much! :)

Should probably have a script play that on loudspeakers when monitoring detects problems /s

Re: Akamai Edge DNS was down

#219

I ran DNS servers, among other things, in the late 90s with better uptime than these "multi-DC/AZ/geo redundant" services everyone uses these days.

With all due respect, having also run auth DNS servers in the 90s, and seen the inside of Akamai’s CDN/DNS setup more recently, it isn’t remotely at the same level of scale or sophistication.

"Scale and sophistication" scale relatively with time. Those servers we ran were relatively at the same level of scale and sophistication for their time. The only differentiator here is uptime, which has gotten worse as time has gone on. Five 9s used to be the standard. Three 9s seems to be the new standard.

Re: Akamai Edge DNS was down

#220
post #49

Earlier quoted context omitted.

Depending on what point you draw the line of "single point of failure" you could use multiple providers for your dns. GOV.UK for example uses both aws and gcp for DNS

Last time I tried setting NS to both cloudflare and digital ocean in my domain registry, cloudflare sent me an email saying the configuration is invalid and asked me to revert. Am I doing something wrong?

No, you have done everything right. At least from the point of view of DNS. That you can not use multiple nameservers is a limitation of Cloudflare (limit in the sense of: Cloudflare can only offer their services in the Free and Pro plan if they have full control over all nameservers).
Post reply on HN