This is extremely concerning as somebody looking to move or build on top of GCP for the long term. I wonder why anyone would choose GCP if outages are occurring on a regular basis.
Google cloud outage
31–39 of 39 posts
Re: Google cloud outage
#32This is extremely concerning as somebody looking to move or build on top of GCP for the long term. I wonder why anyone would choose GCP if outages are occurring on a regular basis.
Any evidence they happen more frequently that the other clouds?
Re: Google cloud outage
#33Re: Google cloud outage
#34Re: Google cloud outage
#35I was bummed out when Siteground moved all their cloud accounts over G, without telling their customers beforehand
Re: Google cloud outage
#36Earlier quoted context omitted.
Have you worked with redundant routers? They certainly reduce the number of outages, but sometimes the hardware (or software) fails in exciting ways that doesn't engage the redundancy, or doesn't engage it properly, and you still get an outage (or you get an outage that wouldn't have happened). Or sometimes, one circuit is out of service for repair or upgrade, and the other circuit is connected to the router that fai…
I have. I am just highlighting that the problem surely should be more complex than described. Or that their redundancy for these events was not adequately devised.
Google often releases a pretty solid post-mortem, which will give the detail of the event. The level of detail appropriate for same-day release is really 'router failure' or 'power failure' or 'software failure' or 'vehicle drove into the building failure'. Expecting more than 'we know what it was, and we fixed it' or 'we don't know what it was, but it stopped happening' or 'yes, we're working on it' on a same-day twitter post is unreasonable.
Re: Google cloud outage
#37Earlier quoted context omitted.
Was it like... a hardware failure? If you serve more than 100 people you probably should have redundant routers. Was it a configuration issue that replicated over to multiple devices at least, I hope?
Have you worked with redundant routers? They certainly reduce the number of outages, but sometimes the hardware (or software) fails in exciting ways that doesn't engage the redundancy, or doesn't engage it properly, and you still get an outage (or you get an outage that wouldn't have happened). Or sometimes, one circuit is out of service for repair or upgrade, and the other circuit is connected to the router that fai…
I remember seeing a Security Analyst for the DC I worked for take down 6 racks worth of Cisco Catalyst 12000 series router hardware once.
They had a HSRP interface set up at the .1 address, and the security analyst set his laptop up with the same static .1 IP address and plugged it in. Instant outage.
Re: Google cloud outage
#38Earlier quoted context omitted.
The bigger issue is that there's a lot of customers where they have split cloud deployments, which means the customers hurt even if they are stable within the clouds themselves.
If you are deployed in such a way that both GCP and AWS need to be up you're doing it backwards. Multi-cloud strategy is supposed to result in the intersection of cloud failures, not the union of them.
I have heard that many companies are multi cloud as a result of acquisitions, resulting in a dependency on both clouds.
Re: Google cloud outage
#39I feel bad for these devs. For years they went around proclaiming they had great engineers, but not many people saw their tech, so nobody could verify this.
Then they made one engineering product with public customers and it fails miserably compared to the competition.
That must hurt on the inside knowing they are frauds.