Live data from Hacker News

Google Cloud Is Down

news.ycombinator.com

361–370 of 630 posts

Re: Google Cloud Is Down

#361

And thus was ruined hundreds or thousands of pleasant Sunday afternoons. I don’t miss being on pager duty one bit. I see it looming in my headlights, sadly.

It's the Queen's birthday, a Monday off here in New Zealand... ... but not for everybody now.

So what happens when the crown changes? They change the holiday? Immediately? For the next year? Sounds like a bit of a nightmare.

Re: Google Cloud Is Down

#363
post #280

It seems crazy to me that Google Cloud can have this level of instability but I, on the other hand, can never remember google.com going down. Why are they operating one with a different networking infrastructure from the other?

They did have a major outage for a few minutes on 2013: https://www.theregister.co.uk/2013/08/17/google_outage/

So this is now the longest outage ever, right?

Re: Google Cloud Is Down

#364

Earlier quoted context omitted.

It seems global to me. This is really strange compared to AWS. I don't remember an outage there (other than s3) impacting instances or networking globally.

I have an instance in us-west-1 (Oregon) which is up, but an instance in us-west-2 (Los Angeles) which is down. Not sure if that means Oregon is unaffected though.

us-west-1 is Northern California (Bay area). us-west-2 is Oregon (Boardman).

Re: Google Cloud Is Down

#365
post #345

This must be why GitLab is giving me shit. They recently made the switch. Wonder if there is any second guessing going on over there right now.

They have a incident report page up at: https://docs.google.com/document/d/1RM3QnuJ4FPH10J3UrJS0T26d...

Using Google Docs to report problems with Google, including Google Docs. Irony at it's finest.

Re: Google Cloud Is Down

#366
Looks like this is why Shopify servers are down as well. Apparently last year they moved everything to GCP.

I don't know why anyone uses them. They have proven to be highly unreliable and have horrible support.

While I'm at it, I'm still bitter about gitlab switching to gcp from azure. They were trying to save face after the github acquisition and just moved on to an even more dubious company for their servers.

Re: Google Cloud Is Down

#367

Yep, I can no longer see my Cloud SQL database - it's as if I've never created one at all. Really hoping this is just an issue displaying it and that Google hasn't punted my infrastructure and backups.

[I am the Cloud SQL tech lead]

This is a networking issue, and your data is safe. Cloud SQL stores instance metadata regionally, so it shares a failure domain with the data it describes. When the region is down or inaccessible, instances are missing from the list results, but that doesn't say anything about the instance availability from within region.

Re: Google Cloud Is Down

#368
post #13
post #6

That feeling when you open https://console.cloud.google.com and see that you don't have your Kubernetes clusters and CloudSQL databases, but CTA to create first.

Gosh, this was so scary... I thought someone had hacked in and deleted everything... I hope they come back. This is still pretty scary

[I am the Cloud SQL tech lead]

This is a networking issue, and your data is safe. Cloud SQL stores instance metadata regionally, so it shares a failure domain with the data it describes. When the region is down or inaccessible, instances are missing from the list results, but that doesn't say anything about the instance availability from within region.

Re: Google Cloud Is Down

#369
post #345

Earlier quoted context omitted.

They have a incident report page up at: https://docs.google.com/document/d/1RM3QnuJ4FPH10J3UrJS0T26d...

Using Google Docs to report problems with Google, including Google Docs. Irony at it's finest.

They do have an internal ticket as well, but Gitlab.com is also hosted on Google, so... https://gitlab.com/gitlab-com/gl-infra/production/issues/862

Re: Google Cloud Is Down

#370
post #344

Earlier quoted context omitted.

I am not sure if a single S3 outage pushed any big names into their own "datacenter". S3 has still the world record of reliability that you cannot challenge with your inhouse solutions. You can prove it otherwise. I would love to hear a solution that has the same durability, avabiality and scalability as S3. For the downvoters, please just link here the proof if you disagree. Here are the S3 numbers: https://aws.amaz…

> For the downvoters, please just link here the proof if you disagree. https://wasabi.com/

How can they possibly guarantee eleven nines? Considering I’ve never heard of this company and they offer such crazy-sounding improvements over the big three, it feels like there should be a catch.
Post reply on HN