Live data from Hacker News

Akamai Edge DNS was down

edgedns.status.akamai.com

91–100 of 229 posts

Re: Akamai Edge DNS was down

#91
post #46
post #29

Was just browsing a website where the first page of a query worked, but visiting page 2 of the results was returning a DNS error. Was curious how and why only part of the site was down, but it looks like this was the problem as now the whole site is down.

aren't short DNS TTLs great?

Is this a serious argument for long TTLs? Always wondered why they exist… How interesting.

Re: Akamai Edge DNS was down

#92
post #49

Earlier quoted context omitted.

Depending on what point you draw the line of "single point of failure" you could use multiple providers for your dns. GOV.UK for example uses both aws and gcp for DNS

So, NS entries pointing to both? But then take the example your domain was in Route53 and AWS goes down. You can't configure the NS entries to avoid AWS DNS servers. Is the idea that child DNS servers detect the outage and cache the values in the name server(s) that remain up? But then, the cached values from AWS take a while to clear, TTL never seems to be applied properly. It always feels like the worst case in suc…

Ibthink if route53 was down. Your dns provider whouldn't able to go there. So it will go to the root who will give gcp one too. So your dns provider might try that.

(I don't know if this is how it works, but I thibk that's how it supposed to work)

Re: Akamai Edge DNS was down

#93
post #86
post #26

Earlier quoted context omitted.

It's an interesting question, as it's always been solved on the server side. All of the current problem is client side. That is, client resolvers that aren't using diverse providers, and only do things like round-robin with long timeouts.

Anycast for the DNS IPs deals with most of the problems of clients not failing over elegantly when their primary DNS server is broken.

From a client (DNS recursor) point of view there is no primary server. There is just multiple NS records which are equal. If one of them is down it can introduce resolving delays, but they are usually small. At least if something like Unbound or Bind is used. Unbound e. g. maintains infra-cache where it tracks RTT and errors for each server and avoid servers which are down.

Re: Akamai Edge DNS was down

#97
post #49

Earlier quoted context omitted.

Depending on what point you draw the line of "single point of failure" you could use multiple providers for your dns. GOV.UK for example uses both aws and gcp for DNS

So, NS entries pointing to both? But then take the example your domain was in Route53 and AWS goes down. You can't configure the NS entries to avoid AWS DNS servers. Is the idea that child DNS servers detect the outage and cache the values in the name server(s) that remain up? But then, the cached values from AWS take a while to clear, TTL never seems to be applied properly. It always feels like the worst case in suc…

You set both services in your ns records. So every day they share the load for dns resolution. If one day one of them is down the client can/will use a different nameserver from your configuration.

Re: Akamai Edge DNS was down

#98
post #55

I am surprised financial institutions don't have any regulation for redundancy. The one that stuck out to me is the Navy Federal Credit Union website being down. I have not had any issues logging into mobile though for some of the reported sites.

Commercial banks are held to a different operational resiliency standard than financial infrastructure.

(a component of my consulting work is reporting to financial regulators for institutions)

Re: Akamai Edge DNS was down

#99
post #7

You can see this on a lot of sites right now. You get the Akamai style error with something like: Reference: #11.453a2f17.1393u44848484.3aee33433 At the bottom of a very bland looking error page.

What's frustrating is that DNS is returning an address, instead of just failing, and so macos is caching that value (though it might be cloudflare doing that).

Wildcard DNS should be a prosecutable crime, punishable by no less than 20 years of hard labor. (Edit: Probably should have made it clear that this was a joke)

Re: Akamai Edge DNS was down

#100
post #49

Earlier quoted context omitted.

Depending on what point you draw the line of "single point of failure" you could use multiple providers for your dns. GOV.UK for example uses both aws and gcp for DNS

So, NS entries pointing to both? But then take the example your domain was in Route53 and AWS goes down. You can't configure the NS entries to avoid AWS DNS servers. Is the idea that child DNS servers detect the outage and cache the values in the name server(s) that remain up? But then, the cached values from AWS take a while to clear, TTL never seems to be applied properly. It always feels like the worst case in suc…

Configuring two NS entries is pretty standard, so surely most resolvers try one of the two, and if it's down try the other one? What else would be the point of having multiple nameservers? Then you just have to get two nameserver providers and make sure their settings stay synced, and point your domain to one nameserver from each.

Of course that requires the server to properly fail, i.e. stop responding to requests. That doesn't seem to be the case here

Post reply on HN