GoDaddy outage caused by corrupted router tables

Post by upgray-d » Tue, Sep 11, 2012, 6:14 PM UTC

gvb wrote: post #18

sophacles wrote: post #14

Earlier quoted context omitted.

Routers at this level aren't just scale-ups of your home wifi/nat box. They aren't even scale ups of the simple IP routers for a basic IT data-closet that manages subnets and whatnot (already much more complex by dealing with vlans and subnets and dmz and vpn issues). At the level of big networking company they are a truly complex beast. Just at the IP level they have to deal with (at the edges and across substantial…

...and if you don't have up to date backups of your router tables, it will take a long time to recover from an "oops". Doing the wrong thing to router table(s) is the network equivalent of "sudo rm -rf /".

Post by noselasd » Tue, Sep 11, 2012, 6:26 PM UTC

overworkedasian wrote: post #11

I wasnt really paying attention to the outage, but if it was indeed a routing issue, then you shouldnt have been able to reach any godaddy ip address. ICMP/traceroutes would have failed and showed the error.

Post by jpdoctor » Tue, Sep 11, 2012, 6:34 PM UTC

dclowd9901 wrote: post #31

staunch wrote: post #10

Based on a long history of working in datacenters I'd bet someone misconfigured something and later claimed it was "corrupted" to save their ass - happens all the time. It's just so simple to make very confusing and damaging mistakes in a complicated network. I wouldn't be surprised to hear that GoDaddy's corporate culture wouldn't respond well to someone admitting to a mistake this damaging.

Ex-GoDaddy employee here: Everyone there is on pins and needles at this point. Since Silverlake's investment in the company, many hatchets have dropped on jobs, and it's really the only decent tech firm in Phoenix to work at. My guess is that there is some hiney covering going on with this explanation, and the interim CEO has little cause to care too much about responsibility, since he'll likely be out before year's…

Post by 46Bit » Tue, Sep 11, 2012, 6:39 PM UTC

andreasvc wrote: post #46

sendos wrote: post #7

Earlier quoted context omitted.

> So Anonymous0wn3r or whoever was just claiming responsibility for something they had no hand in? Yeah, with a name/handle like Anonymous0wn3r they sound very trustworthy. If they claimed they had a hand in something, it must be true.

But didn't he announce the attack before-hand?

Post by stef25 » Tue, Sep 11, 2012, 6:41 PM UTC

kevincennis wrote: post #50

For anyone interested, the person who claimed responsibility for this is tweeting about GoDaddy's response: https://twitter.com/AnonymousOwn3r/status/245568841160196096