Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Stack Overflow plans to survive the next DNS attack

blog.serverfault.com

Post reply on HN
Print view View original

1–10 of 41 posts

Re: How Stack Overflow plans to survive the next DNS attack

#2

Post by bks »

Umm, brilliant thank you for this.

I ended up with a Dyn / Route53 configuration. We used libcloud to sync everything together. We also added the exported zone to Cloudflare but did not enable it.

We had actually planned for this, but in no way did we ever come close to your in depth testing. The @ Azure issue - thank you for uncovering this for the rest of us.

Re: How Stack Overflow plans to survive the next DNS attack

#5

Post by skuhn »

cuu508 wrote: post #4

Is there a good writeup somewhere about setting up redundant NS records at the zone apex? Or, more generally, "DNS primer for busy developer" article?

Once you have configured your zone with multiple providers, it's simply a matter of adding NS entries for each provider's authoritative servers to your registrar. The harder part is ensuring that the zones are kept in sync and that you don't rely on features (such as GSLB stuff or ALIAS records) that aren't available with all providers.

It's up to the client resolver to handle failover, so it's not perfect in terms of availability, but better than nothing.

For example:

  $ dig ns amazon.com
  amazon.com.		3599	IN	NS	ns4.p31.dynect.net.
  amazon.com.		3599	IN	NS	ns1.p31.dynect.net.
  amazon.com.		3599	IN	NS	ns3.p31.dynect.net.
  amazon.com.		3599	IN	NS	ns2.p31.dynect.net.
  amazon.com.		3599	IN	NS	pdns1.ultradns.net.
  amazon.com.		3599	IN	NS	pdns6.ultradns.co.uk.
(note that this is also TLD redundant, since there's a .co.uk included)

Re: How Stack Overflow plans to survive the next DNS attack

#7

Post by captncraig »

elktea wrote: post #3

Netflix have a tool for this as well https://github.com/Netflix/denominator

We have a pretty cool tool for managing this at Stack. Changing from a single provider to two can literally be done by changing one line of config. We should be open sourcing it very soon, and a blog post on it sometime thereafter.

Re: How Stack Overflow plans to survive the next DNS attack

#8

Post by captncraig »

skuhn wrote: post #5
cuu508 wrote: post #4

Is there a good writeup somewhere about setting up redundant NS records at the zone apex? Or, more generally, "DNS primer for busy developer" article?

Once you have configured your zone with multiple providers, it's simply a matter of adding NS entries for each provider's authoritative servers to your registrar. The harder part is ensuring that the zones are kept in sync and that you don't rely on features (such as GSLB stuff or ALIAS records) that aren't available with all providers. It's up to the client resolver to handle failover, so it's not perfect in terms o…

The tricky part is making sure the apex NS records are consistent across all authoritative nameservers. A surprising number of dns providers do not let you edit those.

Re: How Stack Overflow plans to survive the next DNS attack

#9

Post by skuhn »

captncraig wrote: post #8
skuhn wrote: post #5

Earlier quoted context omitted.

Once you have configured your zone with multiple providers, it's simply a matter of adding NS entries for each provider's authoritative servers to your registrar. The harder part is ensuring that the zones are kept in sync and that you don't rely on features (such as GSLB stuff or ALIAS records) that aren't available with all providers. It's up to the client resolver to handle failover, so it's not perfect in terms o…

The tricky part is making sure the apex NS records are consistent across all authoritative nameservers. A surprising number of dns providers do not let you edit those.

Yeah, it does need to be the same in the zone as well as with your registrar. As mentioned in StackOverflow's blog post, Azure doesn't support changing NS records.

Similarly, there are a fair number of DNS providers that don't allow you to use all DNS record types. For something so simple, providers can really go out of their way to screw it up.

Re: How Stack Overflow plans to survive the next DNS attack

#10

Post by paulddraper »

skuhn wrote: post #6
elktea wrote: post #3

Netflix have a tool for this as well https://github.com/Netflix/denominator

Denominator is not actively developed: https://github.com/Netflix/denominator/issues/374 Last commit of substance was in Sept 2015.

Weird. I attended a lecture about it not too long ago.

I wonder what Netflix is doing instead.

Post reply on HN