[deleted]
Google Cloud region currently down due to water intrusion
21–30 of 187 posts
Re: Google Cloud region currently down due to water intrusion
#22Earlier quoted context omitted.
There's an interesting Twitter thread about that topic here: https://twitter.com/GergelyOrosz/status/1651256082424012806 Based on that thread it sounds like only AWS guarantees that their AZs are in physically separate DCs, while for Google and Microsoft AZs could be in separate buildings of the same DC facility.
You would think that the company that literally wrote the book on “Site Reliability Engineering” would actually follow their own recommendations.
They do internally. But when customers want 3 zones in Indonesia they cut corners.
Re: Google Cloud region currently down due to water intrusion
#23[flagged]
Re: Google Cloud region currently down due to water intrusion
#24Anyone know how this could affect multiple zones? "Customers can failover to zones in other regions". Unless a whole area got flooded.
[deleted]
Re: Google Cloud region currently down due to water intrusion
#25Earlier quoted context omitted.
There's an interesting Twitter thread about that topic here: https://twitter.com/GergelyOrosz/status/1651256082424012806 Based on that thread it sounds like only AWS guarantees that their AZs are in physically separate DCs, while for Google and Microsoft AZs could be in separate buildings of the same DC facility.
AWS treats its availability zones very seriously, each zone has its own independent power substation, air conditioning, and fiber lines. It's incredibly rare for multiple AZs to go down at once, especially since they are more than a few miles apart from each other.
Funnily enough floods (GCP) and fires (OVH) are two of the 3 things AWS explicitly mentions in the Well Architected docs. For a lot of companies an AZ going down is an annoyance or bad day but a whole region going down could be a real continuity risk.
> Each Availability Zone is separated by a meaningful physical distance from other zones to avoid correlated failure scenarios due to environmental hazards like fires, floods, and tornadoes.
https://docs.aws.amazon.com/wellarchitected/latest/reliabili...
Re: Google Cloud region currently down due to water intrusion
#26Earlier quoted context omitted.
There's an interesting Twitter thread about that topic here: https://twitter.com/GergelyOrosz/status/1651256082424012806 Based on that thread it sounds like only AWS guarantees that their AZs are in physically separate DCs, while for Google and Microsoft AZs could be in separate buildings of the same DC facility.
AWS treats its availability zones very seriously, each zone has its own independent power substation, air conditioning, and fiber lines. It's incredibly rare for multiple AZs to go down at once, especially since they are more than a few miles apart from each other.
It’s more than that. AZ’s are geographically distinct as well along multiple dimensions include flood plains etc.
Re: Google Cloud region currently down due to water intrusion
#27Last time they let Bard pick a data center location and design.
Re: Google Cloud region currently down due to water intrusion
#28Anyone know how this could affect multiple zones? "Customers can failover to zones in other regions". Unless a whole area got flooded.
There's an interesting Twitter thread about that topic here: https://twitter.com/GergelyOrosz/status/1651256082424012806 Based on that thread it sounds like only AWS guarantees that their AZs are in physically separate DCs, while for Google and Microsoft AZs could be in separate buildings of the same DC facility.
Yes. Azure and GCPs numbers on the size of their AZs and such are more marketing spin than hard engineering. AWS keeps these in separate physical locations to provide true separation. While there have been tech related regional incidents at AWS a physical event disabling multiple AZs would be extremely unlikely given their much more robust and geographically distributed design. If such a physical event had happened in AWS it would have been a non-event with things just failing over to other AZs.
Other cloud providers mostly just vaguely put things in another part of the building and say it’s “a separate AZ” but as GCPs woes highlighted that’s corner cutting that bites badly when the whole building has a problem.
Re: Google Cloud region currently down due to water intrusion
#29This is the problem storing data in the cloud - whenever it rains you may have a big data problem.