Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Cloud region currently down due to water intrusion

status.cloud.google.com

Post reply on HN
Print view View original

161–170 of 187 posts

Re: Google Cloud region currently down due to water intrusion

#161

Post by riceart »

dsfyu404ed wrote: post #146
local_crmdgeon wrote: post #122

Earlier quoted context omitted.

Firefighters are personally incentivized to take photos like this: https://nypost.com/2022/01/10/fireman-in-post-photo-recalls-... . They are not incentivized to protect your stuff, that's what insurance does. Good. I want you to save me and my family from dying in a fucking fire. This thread is obsessed with saving Funko Pop collections for some reason??

That's just a textbook appeal to emotion. Most of their calls are mundane stuff. And they leave a pretty decently wide path of destruction in doing that. We're talking like mundane situations where there is no urgency and no need to tear shit up in the interest of time. I once arrived to a minor rollover after the cops but before fire. Nobody injured. Occupant trapped because she was a large lady and couldn't release…

You have any real data to back this up rather than some anecdotes?

You sound really butthurt.

Re: Google Cloud region currently down due to water intrusion

#164

Post by sbierwagen »

local_crmdgeon wrote: post #122
dsfyu404ed wrote: post #112

Earlier quoted context omitted.

Anyone talking up firefighters like this clearly hasn't been around them much. They're boys with toys that they don't frequently get to use and they work for the government. Follow the incentives. They'll do their jobs but they don't give a lot of fucks about things like "unnecessary property damage" and "other people's financial well being" and anything else not written in their KPIs. I used to drive tow truck. I ca…

Firefighters are personally incentivized to take photos like this: https://nypost.com/2022/01/10/fireman-in-post-photo-recalls-... . They are not incentivized to protect your stuff, that's what insurance does. Good. I want you to save me and my family from dying in a fucking fire. This thread is obsessed with saving Funko Pop collections for some reason??

Hello 16 day old account. Can't imagine why you've been banned before. https://news.ycombinator.com/item?id=35714352

Re: Google Cloud region currently down due to water intrusion

#166

Post by 1letterunixname »

t0mas88 wrote: post #46

I can't ignore the feeling that Google Cloud is sub par compared to AWS. How did this again cause a multi zone failure. Why haven't they fixed those dependencies the last few times they had a full region failure.

Well, GCP did fire most of their senior customer techs 18 months ago.

Alphabet is most ruled by arbitrary fiat of empire-builders.

Killing products and footgun changes are standard.

Re: Google Cloud region currently down due to water intrusion

#167

Post by dx034 »

packetslave wrote: post #139
glogla wrote: post #127

Earlier quoted context omitted.

Yeah, I always thought datacenters would use Halon. It of course has the problem of suffocating everyone.

Halon has been banned for years because 1) it's bad for the ozone layer and 2) it'll kill you. Newer systems (FM-200, Inergen, etc.) fight the fire by removing heat instead of removing oxygen.

Wasn't there also this technique of lowering oxygen levels so much that humans can still survive but fire won't spread as fast? Or did this turn out to be too expensive?

Re: Google Cloud region currently down due to water intrusion

#168

Post by madaxe_again »

jacquesm wrote: post #137
madaxe_again wrote: post #135

Earlier quoted context omitted.

Oh, the irony. A few years ago I implemented a top to bottom ISO27k1 ISMS for a client handling extremely sensitive and mission-critical data for industry. One risk I recommended controls for was that of a fire and/or flood at their primary datacentre for their client-facing offerings - this datacentre. I’ve experienced the misery of a datacentre oops myself, firsthand, twice, and it’s a genuine risk that has to be m…

There's that beautiful German word again... schadenfreude. I have had similar discussions multiple times in the last year and the magic thinking around the cloud is so strong that it is sometimes impossible to get through. The fact that cloud stuff can go down and that in the end it is your data and no amount of cloud credits are going to help you if your data is lost seems to be utterly beyond some people's comprehe…

In this case it wasn’t so much “the cloud is invincible!” as “this appears to be a cost centre” - their whole shtick is to boost profitability in the short term by gutting businesses, and then selling them onwards to some other finance sucker.

I am sure that my name is currently being damned in a boardroom in Chicago, as as the person who warned of this, I am likely seen as responsible.

Re: Google Cloud region currently down due to water intrusion

#169

Post by packetslave »

Aeolun wrote: post #158
packetslave wrote: post #141

Earlier quoted context omitted.

A non-water fire suppression system for a 300,000+ square feet warehouse-scale datacenter would be incredibly expensive.

Would it be as expensive as replacing 300,000+ square feet of servers?

People who aren't me get paid to do that kind of math when designing datacenters. Presumably there's a break-even point.

Re: Google Cloud region currently down due to water intrusion

#170

Post by mike_hearn »

stingraycharles wrote: post #124
nik736 wrote: post #18

Earlier quoted context omitted.

Well, we had pictures very quickly of the OVH fire. Google seems to be not very transparent on what is exactly happening...

The linked article says that there was a leak in a water cooling system, which in turn ended up in the battery system which caused a fire. But yeah it’s not coming from Google but second hand reports.

It's not second hand, it's from the colo provider they're using in Paris.
Post reply on HN