Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DNS Outage at DigitalOcean

status.digitalocean.com

Post reply on HN
Print view View original

81–90 of 125 posts

Re: DNS Outage at DigitalOcean

#81

Post by zackelan »

tonylemesmer wrote: post #13

People hating on DO "I'm losing thousands every hour". Well then should have had some failover in place if its that valuable. [1] https://twitter.com/rodrigoespinosa/status/71303563702097100...

For an example of this taken to a ludicrous extreme, several years ago an AWS user complained that downtime of a few EC2 instances were putting lives at risk. They were hosting cardiac monitoring services on single EC2 instances with no multi-AZ or multi-region capability.

AWS forum post: https://forums.aws.amazon.com/thread.jspa?threadID=65649&tst...

Previous HN discussion: https://news.ycombinator.com/item?id=2477345

Re: DNS Outage at DigitalOcean

#82

Post by takeda »

jbaptiste wrote: post #53
bitJericho wrote: post #36

Earlier quoted context omitted.

That's why you use multiple providers.

Even with multi providers, DNS issues are a cluster fuck.

Only if you don't know what you're doing. The problem with DNS is that it might work even when it is misconfigured, and misconfiguration is the source of strange issues.

Re: DNS Outage at DigitalOcean

#83

Post by takeda »

coreyp_1 wrote: post #7

Does anyone know of a good strategy for DNS failover?

That's one of the easiest thing to do. Just add multiple NS to the domain. As long as they are configured correctly and at least one is up (and for limited time even if none are up) the service is available.

With DNS you actually can achieve 100% uptime.

Re: DNS Outage at DigitalOcean

#84

Post by johansch »

takeda wrote: post #80
johansch wrote: post #76

Earlier quoted context omitted.

BS. DNS is a trivial thing to scale, compared to most other web-scale efforts. Things break when people don't use 20 year old best practices. There is no defense against inexperience and ignorance.

I took the OPs comment as "it's hard to understand DNS and biggest fuck ups happen because people think they understand DNS when they actually don't". The problem with DNS is that it can work even when it is configured incorrectly. This makes people who has no idea what they are doing that they actually understand it. The strange issues with DNS only happen with strange configurations. When you follow best practices…

All right. This I can agree with.

Re: DNS Outage at DigitalOcean

#85

Post by ludbb »

colinbartlett wrote: post #73
clentaminator wrote: post #62

Earlier quoted context omitted.

But who monitors the status of statuspage.io?

I know you're joking, but I do: Check out https://StatusGator.com . StatusPage.io has a status page at metastatuspage.com which my company monitors.

And who monitors your company monitors? :)

Re: DNS Outage at DigitalOcean

#86

Post by sashk »

copperx wrote: post #68
sashk wrote: post #15

My rule: provider should do single thing: - Hosting provider - host sites - vps/cloud provider - provide VMs - domain registrar - domain related stuff, but not DNS - dns provider - host dns - second dns provider - host dns in case first dns provider fails So many DNS outages recently and all my projects are up.

Does Amazon's Route 53 count as a DNS provider, or do you treat it a hosting provider?

For me - neither.

But if I'd be tied in into Amazon's cloud infrastructure, I would have to use many of their features going against my rules above.

Re: DNS Outage at DigitalOcean

#87

Post by camikazeg »

colinbartlett wrote: post #75

If you want to get alerted when it comes back up, or you wish you had been alerted when it went down, check out my project: https://StatusGator.com . StatusGator monitors status pages and sends notifications via email, Slack, and others. You can get alerted to status changes inside Slack and you can ask it the status of a service with a /statuscheck command.

A bit of feedback: you should have a link back to your dashboard on every page. That seems like the most important page to me as a user, but if I am changing my notification or account settings, there is no way back to that page.

Re: DNS Outage at DigitalOcean

#88

Post by ludbb »

sashk wrote: post #86
copperx wrote: post #68

Earlier quoted context omitted.

Does Amazon's Route 53 count as a DNS provider, or do you treat it a hosting provider?

For me - neither. But if I'd be tied in into Amazon's cloud infrastructure, I would have to use many of their features going against my rules above.

How do you apply your rules considering what's available today? Which services are you using? It sounds like it would be a big headache to orchestrate the automation among all these different providers.

Re: DNS Outage at DigitalOcean

#90

Post by camikazeg »

takeda wrote: post #82
jbaptiste wrote: post #53

Earlier quoted context omitted.

Even with multi providers, DNS issues are a cluster fuck.

Only if you don't know what you're doing. The problem with DNS is that it might work even when it is misconfigured, and misconfiguration is the source of strange issues.

I think that we all have areas where we don't know what we're doing. This is one of mine. With all the talk of how obvious/important/easy it is to have a failover in place in case this happens, I'm having trouble finding a good resource about setting up a redundant DNS. Running a droplet on Digital Ocean with Debian and Nginx.
Post reply on HN