Massive Dyn DNS outage • Hacker News Board

Re: Massive Dyn DNS outage

#181

Post by drmunklung » Fri, Oct 21, 2016, 5:21 PM UTC

deno wrote: post #64

num3thod wrote: post #18

I just realised how online adult entertainment has the most redundancy of any Internet service category, bar none.

You’re right :) $ dig @8.8.8.8 ns +short pornhub.com ns2.p44.dynect.net. ns3.p44.dynect.net. ns4.p44.dynect.net. sdns3.ultradns.net. sdns3.ultradns.com. sdns3.ultradns.org. sdns3.ultradns.biz. ns1.p44.dynect.net. PornHub has better DNS redundancy than Github and Twitter.

I'm sure this is to circumvent blacklisting rather than for this scenario.

Re: Massive Dyn DNS outage

#182

Post by Sami_Lehtinen » Fri, Oct 21, 2016, 5:23 PM UTC

Namecheap is also down.

Re: Massive Dyn DNS outage

#183

Post by jcl » Fri, Oct 21, 2016, 5:25 PM UTC

mikemcquaid wrote: post #8

GitHub employee here. We're monitoring an incident with our upstream DNS provider: https://twitter.com/githubstatus/status/789433336083001344

This link seems to work for me to reach the github status page (requires https certificate override, of course):

https://107.22.212.99/

(Alias for https://status.github.com, via https://octostatus-9676240.us-east-1.elb.amazonaws.com )

Re: Massive Dyn DNS outage

#184

Post by drcoopster » Fri, Oct 21, 2016, 5:26 PM UTC

Mingso wrote: post #152

It's 9:55am PST California and Twitter is down.

Did you change your clocks early?

Re: Massive Dyn DNS outage

#185

Post by DonaldFoss » Fri, Oct 21, 2016, 5:27 PM UTC

jbaviat wrote: post #124

Issue came from dynect.net: $ dig +trace github.com ; > DiG 9.8.3-P1 > +trace github.com [...] ;; Received 488 bytes from 192.228.79.201#53(192.228.79.201) in 154 ms github.com. 172800 IN NS ns1.p16.dynect.net. github.com. 172800 IN NS ns3.p16.dynect.net. github.com. 172800 IN NS ns2.p16.dynect.net. github.com. 172800 IN NS ns4.p16.dynect.net. ;; Received 178 bytes from 192.42.93.30#53(192.42.93.30) in 54 ms ;; conne…

As of 2016-10-21T09:20:19Z, it's doing better: -------------------------------------------------------------------- :$ dig +trace github.com

; > DiG 9.8.3-P1 > +trace github.com ;; global options: +cmd . 17430 IN NS i.root-servers.net. . 17430 IN NS a.root-servers.net. . 17430 IN NS b.root-servers.net. . 17430 IN NS l.root-servers.net. . 17430 IN NS d.root-servers.net. . 17430 IN NS g.root-servers.net. . 17430 IN NS h.root-servers.net. . 17430 IN NS c.root-servers.net. . 17430 IN NS e.root-servers.net. . 17430 IN NS j.root-servers.net. . 17430 IN NS k.root-servers.net. . 17430 IN NS m.root-servers.net. . 17430 IN NS f.root-servers.net. ;; Received 228 bytes from 8.8.8.8#53(8.8.8.8) in 382 ms

com. 172800 IN NS m.gtld-servers.net. com. 172800 IN NS k.gtld-servers.net. com. 172800 IN NS a.gtld-servers.net. com. 172800 IN NS b.gtld-servers.net. com. 172800 IN NS f.gtld-servers.net. com. 172800 IN NS j.gtld-servers.net. com. 172800 IN NS e.gtld-servers.net. com. 172800 IN NS c.gtld-servers.net. com. 172800 IN NS h.gtld-servers.net. com. 172800 IN NS l.gtld-servers.net. com. 172800 IN NS d.gtld-servers.net. com. 172800 IN NS g.gtld-servers.net. com. 172800 IN NS i.gtld-servers.net. ;; Received 488 bytes from 192.228.79.201#53(192.228.79.201) in 363 ms

github.com. 172800 IN NS ns1.p16.dynect.net. github.com. 172800 IN NS ns3.p16.dynect.net. github.com. 172800 IN NS ns2.p16.dynect.net. github.com. 172800 IN NS ns4.p16.dynect.net. ;; Received 178 bytes from 192.42.93.30#53(192.42.93.30) in 134 ms

github.com. 300 IN A 192.30.253.113 github.com. 86400 IN NS ns4.p16.dynect.net. github.com. 86400 IN NS ns3.p16.dynect.net. github.com. 86400 IN NS ns1.p16.dynect.net. github.com. 86400 IN NS ns2.p16.dynect.net. ;; Received 130 bytes from 2001:500:90:1::16#53(2001:500:90:1::16) in 25 ms

I ended up sending all my developers a patch to their hosts file... Hopefully they'll roll it back shortly.

--Donald

Re: Massive Dyn DNS outage

#186

Post by dtoombs » Fri, Oct 21, 2016, 5:28 PM UTC

deno wrote: post #64

num3thod wrote: post #18

I just realised how online adult entertainment has the most redundancy of any Internet service category, bar none.

You’re right :) $ dig @8.8.8.8 ns +short pornhub.com ns2.p44.dynect.net. ns3.p44.dynect.net. ns4.p44.dynect.net. sdns3.ultradns.net. sdns3.ultradns.com. sdns3.ultradns.org. sdns3.ultradns.biz. ns1.p44.dynect.net. PornHub has better DNS redundancy than Github and Twitter.

Amazon does too:

  C:\>nslookup

  > set type=ns
  > amazon.com

  Non-authoritative answer:
  amazon.com      nameserver = ns3.p31.dynect.net
  amazon.com      nameserver = pdns1.ultradns.net
  amazon.com      nameserver = ns4.p31.dynect.net
  amazon.com      nameserver = pdns6.ultradns.co.uk
  amazon.com      nameserver = ns1.p31.dynect.net
  amazon.com      nameserver = ns2.p31.dynect.net

Re: Massive Dyn DNS outage

#187

Post by moonlightsky » Fri, Oct 21, 2016, 5:29 PM UTC

https://ease.apperian.com is down over here

Re: Massive Dyn DNS outage

#188

Post by [deleted] » Fri, Oct 21, 2016, 5:29 PM UTC

[deleted]

Re: Massive Dyn DNS outage

#189

Post by eric_the_read » Fri, Oct 21, 2016, 5:30 PM UTC

cynosurexy wrote: post #163

jssjr wrote: post #85

I'm a GitHub employee and want to let everyone know we're aware of the problems this incident is causing and are actively working to mitigate the impact. "A global event is affecting an upstream DNS provider. GitHub services may be intermittently available at this time." is the content from our latest status update on Twitter ( https://twitter.com/githubstatus/status/789452827269664769 ). Reposted here since some peo…

how did you post on twitter which is down too?

Cached DNS response.

Re: Massive Dyn DNS outage

#190

Post by mancerayder » Fri, Oct 21, 2016, 5:32 PM UTC

dsl wrote: post #154

mancerayder wrote: post #119

Earlier quoted context omitted.

If ONLY they had 10X developers ... EDIT: joking aside, the issue with multiple DNS providers is primarily (in my experience at the company I'm at having investigated this in the past) intelligent DNS entries. Example, 'return these A records, in this order, based on the number of requests, roughly balanced'. There's no universal standard, just common aspects. DNS Provider A has one set of features, names for returni…

In my experience Dyn pushes this functionality very hard during customer on boarding, effectively as a lock in for their platform.

Well I really think those features are useful, though. They let you get some powerful de-facto load balancing and HA and other complex 'features' without any new hardware or much complexity. They give you some easy creative freedom in designing a redundant infrastructure, give you some geo-aware DNS the likes of which I remember paying Akamai a lot of money for years go, etc. But the lock-in problem is absolutely right.

I'm also familiar with the aggressiveness of that sales team. I prefer another provider and they were trying to solicit our business by specifically calling out our provider as amenable to a DDOS attack which had occurred.

Someone with a simpler setup with standard DNS features will find it much easier to use multiple providers, of course.