Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Google Went Offline Today and a Bit about How the Internet Works

blog.cloudflare.com

Post reply on HN
Print view View original

81–90 of 158 posts

Re: Why Google Went Offline Today and a Bit about How the Internet Works

#81

Post by pilom »

trout wrote: post #33

There are some other ways to fix the problem. Last time with the Youtube problem, they advertised more specific routes. If Pakistan was advertising a /24 network (255 IP addresses) Youtube started advertising two /25 networks (2x 128 addresses). Since they are more specific, they are preferred over the more broad routes. This prevents lack of cooperation, but not malicious behavior. As well, it ends somewhere because…

Much more common that malicious outages is malicious creation of ghost networks. Basically a person could say over BGP "W.X.Y.Z is at my office" where that address isn't used by anyone anywhere else on the internet. Then they do their bad deeds from that made up address. Lastly they remove their route via BGP and it is as if their addresses never existed.

Re: Why Google Went Offline Today and a Bit about How the Internet Works

#82

Post by alok-g »

btilly wrote: post #71
runn1ng wrote: post #51

Can I ask a pretty newbie question - how is BGP connected to IP, TCP and DNS protocols? Is it sitting "below" them, "on top" of them, or is it somewhere else?

Rather than considering them as a meaningful stack, I think it helps to know what each does. IP is a protocol for taking a chunk of data, slapping some addressing information on it, and then having it be sent, like an electronic letter, from one computer to another by whatever route the network thinks is best. More precisely every computer sends it to a computer it is directly connected to that it thinks is closer. E…

Newbie question: If BGP uses TCP to work, TCP is above IP, and routers use BGP information to route IP packets, how may bootstrapping happen if needed?

Re: Why Google Went Offline Today and a Bit about How the Internet Works

#84

Post by BCM43 »

prof_hobart wrote: post #63
bennyg wrote: post #48

Earlier quoted context omitted.

He meant that there is literally only one definition for literally. Any other use of the word cuts away at its meaning (like what happened in that context).

Not really. It is literally something that binds the internet together (i.e. one of the common definitions of the word glue).

Now we can get into what "bind" means.

Re: Why Google Went Offline Today and a Bit about How the Internet Works

#85

Post by Matt_Cutts »

veidr wrote: post #39
amalcon wrote: post #25

Earlier quoted context omitted.

This is basically correct. BGP is weird. The addresses for one of Google's many datacenters were routed incorrectly for packets coming from some subset of IP space. Unless Google is running active ping tests to that subset of IP space, the way they would normally detect it is for someone to call and complain. In this case, the author decided to take a shortcut and call the owner of the "problem peer" directly.

Although only a vanishingly small percentage of Google users can call and complain. Blog or tweet or post to HN and hope Matt Cutts sees it and notifies the right team, maybe.

A team of Googlers could have been working on this in parallel to Tom. I'm guessing that a sudden drop of queries like that would cause people at Google to start digging into what happened. I don't know either way, because network ops and BGP is pretty far from my area (search quality).

Re: Why Google Went Offline Today and a Bit about How the Internet Works

#86

Post by sounds »

pilom wrote: post #81
trout wrote: post #33

There are some other ways to fix the problem. Last time with the Youtube problem, they advertised more specific routes. If Pakistan was advertising a /24 network (255 IP addresses) Youtube started advertising two /25 networks (2x 128 addresses). Since they are more specific, they are preferred over the more broad routes. This prevents lack of cooperation, but not malicious behavior. As well, it ends somewhere because…

Much more common that malicious outages is malicious creation of ghost networks. Basically a person could say over BGP "W.X.Y.Z is at my office" where that address isn't used by anyone anywhere else on the internet. Then they do their bad deeds from that made up address. Lastly they remove their route via BGP and it is as if their addresses never existed.

That might work for some unused /24's for a large organization's /8 block, but unused IPv4 addresses are so last year!

I suppose the attack will still work for IPv6 for a long time.

Re: Why Google Went Offline Today and a Bit about How the Internet Works

#87

Post by jauer »

alok-g wrote: post #82
btilly wrote: post #71

Earlier quoted context omitted.

Rather than considering them as a meaningful stack, I think it helps to know what each does. IP is a protocol for taking a chunk of data, slapping some addressing information on it, and then having it be sent, like an electronic letter, from one computer to another by whatever route the network thinks is best. More precisely every computer sends it to a computer it is directly connected to that it thinks is closer. E…

Newbie question: If BGP uses TCP to work, TCP is above IP, and routers use BGP information to route IP packets, how may bootstrapping happen if needed?

Typically no bootstrapping because IP (and thus TCP) work between hosts in the same subnet without routing.

You would typically have a /30 or /31 subnet containing a pair of routers and have the routers communicate (BGP etc) using those addresses.

Re: Why Google Went Offline Today and a Bit about How the Internet Works

#88

Post by jauer »

killermonkeys wrote: post #3

Why wouldn't PCCW preventing its customers from publishing routes outside its whitelist work? It has been a long time since I worked on BGP but that was common practice from back haul carriers to ISPs even at that point (2003). Given the same back haul provider has allowed this twice, it seems like a reasonable ask.

Some carriers are lazy. There may also be politics involved in making national carriers "ask" for permission to advertise routes.

Re: Why Google Went Offline Today and a Bit about How the Internet Works

#89

Post by jauer »

amalcon wrote: post #25
archangel_one wrote: post #14

Earlier quoted context omitted.

I'm not a network engineer, but it seems like the kind of thing that might be very hard to detect when you're already inside or near to the google.com domain. Or maybe CloudFlare just got there first. I don't think it's necessary to call BS on Cloudflare without any kind of evidence at all.

This is basically correct. BGP is weird. The addresses for one of Google's many datacenters were routed incorrectly for packets coming from some subset of IP space. Unless Google is running active ping tests to that subset of IP space, the way they would normally detect it is for someone to call and complain. In this case, the author decided to take a shortcut and call the owner of the "problem peer" directly.

A common way to notice things like this is to subscribe to a service like Renesys or Cyclops (http://cyclops.cs.ucla.edu/) that will alert you if it sees your subnets being announced by a different AS.
Post reply on HN