Live data from Hacker News

Ask HN: Is Google Compute down?

news.ycombinator.com

31–40 of 80 posts

Re: Ask HN: Is Google Compute down?

#31

Hi all - Seth from Google here. Our team is aware and we are working on mitigation. In short, a third party telco provider is advertising on one of our IP blocks. Unfortunately that's all the information I can share at this time.

EDIT: This is a general statement, I am not complaining to google here.

This kind of thing should not be possible. Are there any protocol proposals or other kind of upgrades to the routing protocols that would prevent these kind of mistakes/attacks?

Re: Ask HN: Is Google Compute down?

#32

I am on the East Coast, in Florida and seeing the same thing with traffic heading to China, lots of "chinatelecom-gw.transtelecom.net" in traceroutes I have never seen prior.

Getting this as well in SF. transtelecom.net WHOIS says they're Moscow-based

Yea, just saw that same thing.

Definitely something interesting going on, and I am sure no shortage of some frantic research and effort to resolve this all at Google and such right now.

Re: Ask HN: Is Google Compute down?

#33

Hi all - Seth from Google here. Our team is aware and we are working on mitigation. In short, a third party telco provider is advertising on one of our IP blocks. Unfortunately that's all the information I can share at this time.

We've updated our status page with as much information as we can provide at this time: https://status.cloud.google.com/incident/cloud-networking/18...

Our teams are continuing to work with upstream and downstream service providers to remedy the issue.

Re: Ask HN: Is Google Compute down?

#35
post #8

Google IPs seem to be being routed to China for us. We have servers in San Jose that cannot access Google services. Trace route shows everything going to China when leaving the San Jose data center. We can access the same services from Vancouver just fine.

How many times does this have to happen before China's privileges to do things like this get revoked? At this point, it can't be just a mistake and must be some state-sponsored hacking. Seems like a great way to find out where a particular Spotify user's IP address is.

Re: Ask HN: Is Google Compute down?

#36

Hi all - Seth from Google here. Our team is aware and we are working on mitigation. In short, a third party telco provider is advertising on one of our IP blocks. Unfortunately that's all the information I can share at this time.

thanks for the update! FWIW we started noticing the connectivity problems around 2018-11-12 21:17 UTC

Thanks for the info. We have reports showing it started a bit earlier than that, but every piece of information is helping in managing an incident. I'll make sure the team is aware.

Re: Ask HN: Is Google Compute down?

#37
post #24
post #8

Google IPs seem to be being routed to China for us. We have servers in San Jose that cannot access Google services. Trace route shows everything going to China when leaving the San Jose data center. We can access the same services from Vancouver just fine.

Seems like its time to start or accelerate a working group on secure BGP.

It's always great to go to your boss/CEO, explain what's going on, and then have to say "no, there is nothing we can do about it."

Re: Ask HN: Is Google Compute down?

#38

Hi all - Seth from Google here. Our team is aware and we are working on mitigation. In short, a third party telco provider is advertising on one of our IP blocks. Unfortunately that's all the information I can share at this time.

EDIT: This is a general statement, I am not complaining to google here. This kind of thing should not be possible. Are there any protocol proposals or other kind of upgrades to the routing protocols that would prevent these kind of mistakes/attacks?

I am surprised how fragile is the internet given how our society is increasingly becoming critically reliant on it.

Re: Ask HN: Is Google Compute down?

#40
Reading through the comments here I'm recognizing "China Telecom" from an article on a BGP hijack that was published about a week ago, I still had the article open in my browser:

https://arstechnica.com/information-technology/2018/11/stran...

In another comment in this thread I read:

> Seems like its time to start or accelerate a working group on secure BGP.

Indeed things can't go on like this for much longer...

Post reply on HN