Live data from Hacker News

How Stack Overflow plans to survive the next DNS attack

blog.serverfault.com

11–20 of 41 posts

Re: How Stack Overflow plans to survive the next DNS attack

#11
I think I measured Cloudflare's performance and chose it over Google because it was consistently faster. If the stack-stackers are reading, I'd love to hear why they didn't make the list.

Also, it'd be a great public service to publish the results. Even if it's just enabled for a day per year or so the results would probably appreciated by many. And you could always sell your altruism as the need to continually monitor the situation :)

Re: How Stack Overflow plans to survive the next DNS attack

#12
The calculation regarding the ideal number of name servers to list needs some empirical data regarding the likelihood of provider and server outages and the client reactions to it, right? Because otherwise 2 would must be the best number, if I'm not mistaken (Chance of hitting the provider that's offline is always 0.5 on the first try, but the second try would be guaranteed to hit the other).

Here's the math for expected number of tries if half of the servers are offline. (It's a hypergeometric distribution but I couldn't find a closed formula)

E(2 server) = 1 * 1/2 + 2 * 1/2 = 1.5

E(4 server) = 1 * 2/4 + 2 * 2/4 * 2/3 + 3 * 2/4 * 1/3 = 1.67

E(8 server) = 1 * 4/8 + 2 * 4/8 * 4/7 + 3 * 4/8 * 3/7 * 4/6 + 4 * 4/8 * 3/7 * 2/6 *4/5 = 1.73

Re: How Stack Overflow plans to survive the next DNS attack

#13

I think I measured Cloudflare's performance and chose it over Google because it was consistently faster. If the stack-stackers are reading, I'd love to hear why they didn't make the list. Also, it'd be a great public service to publish the results. Even if it's just enabled for a day per year or so the results would probably appreciated by many. And you could always sell your altruism as the need to continually monit…

+1 - I am also very interested why they chose to switch out from CloudFlare for both their DNS and their CDN and over to Fastly. Nick Craver did a write up where they specifically mentioned [1] Cloudflare for both their DNS and CDN.

Do you think after the Dyn outage everyones sysadmins are running round adding redundancy, too worried to trust the uptime of their site in the hands of just CloudFlare?

[1] http://nickcraver.com/blog/2016/02/17/stack-overflow-the-arc...

Re: How Stack Overflow plans to survive the next DNS attack

#14
post #2

Umm, brilliant thank you for this. I ended up with a Dyn / Route53 configuration. We used libcloud to sync everything together. We also added the exported zone to Cloudflare but did not enable it. We had actually planned for this, but in no way did we ever come close to your in depth testing. The @ Azure issue - thank you for uncovering this for the rest of us.

Can you maybe share how you did the sync between them? Also there are already some tools pulling zone data from Dyn and adding it to Route53? Can you share why did you choose an own sync. (I'm planning to do the same and I'm interested in other's opinions)

Re: How Stack Overflow plans to survive the next DNS attack

#15
post #6
post #3

Netflix have a tool for this as well https://github.com/Netflix/denominator

Denominator is not actively developed: https://github.com/Netflix/denominator/issues/374 Last commit of substance was in Sept 2015.

I have not looked at this particular commit history, but I want to argue against the notion that no fresh commits must mean that a project is abandoned. Some stuff is just mostly finished after some point.

Re: How Stack Overflow plans to survive the next DNS attack

#16
post #6

Earlier quoted context omitted.

Denominator is not actively developed: https://github.com/Netflix/denominator/issues/374 Last commit of substance was in Sept 2015.

I have not looked at this particular commit history, but I want to argue against the notion that no fresh commits must mean that a project is abandoned. Some stuff is just mostly finished after some point.

In the GitHub issue I linked to, the project maintainer indicates both that he will no longer work on the project and that Netflix has retired the software.

So... I wouldn't invest time in it unless you want to take over stewardship (no one else has offered in the last 7 months).

Re: How Stack Overflow plans to survive the next DNS attack

#17
I'm currently working on a tool [1] that can help with checking if all your different providers are 'in sync' and responding with the same answers. Setups like these are only to grow more common as people realise a single DNS provider is a SPOF of its own.

Very good analysis of SO and a smart move to roll this out _before_ a new DNS outage!

https://dnsspy.io

Re: How Stack Overflow plans to survive the next DNS attack

#18
post #17

I'm currently working on a tool [1] that can help with checking if all your different providers are 'in sync' and responding with the same answers. Setups like these are only to grow more common as people realise a single DNS provider is a SPOF of its own. Very good analysis of SO and a smart move to roll this out _before_ a new DNS outage! https://dnsspy.io

I've thought about doing something like this, the biggest issue I found is feature parity between DNS providers.

If you could have a unified API that would create the records on multiple providers that would be money, it's just that you'd lose out on some things like Route 53 health checking, etc.

Re: How Stack Overflow plans to survive the next DNS attack

#19
It'd be great if more DNS providers supported "slaving" a zone from an existing server. It would make it much easier to keep DNS synchronized across multiple providers.

Hurricane Electric supports this but most of the providers mentioned in this article do not.

Re: How Stack Overflow plans to survive the next DNS attack

#20

I think I measured Cloudflare's performance and chose it over Google because it was consistently faster. If the stack-stackers are reading, I'd love to hear why they didn't make the list. Also, it'd be a great public service to publish the results. Even if it's just enabled for a day per year or so the results would probably appreciated by many. And you could always sell your altruism as the need to continually monit…

+1 - I am also very interested why they chose to switch out from CloudFlare for both their DNS and their CDN and over to Fastly. Nick Craver did a write up where they specifically mentioned [1] Cloudflare for both their DNS and CDN. Do you think after the Dyn outage everyones sysadmins are running round adding redundancy, too worried to trust the uptime of their site in the hands of just CloudFlare? [1] http://nickcr…

tl;dr: Javascript/Google reCaptcha Paywall.

Source:

https://meta.stackoverflow.com/questions/323537/cloudflare-i...

Post reply on HN