Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Cloud region currently down due to water intrusion

status.cloud.google.com

Post reply on HN
Print view View original

111–120 of 187 posts

Re: Google Cloud region currently down due to water intrusion

#111

Post by local_crmdgeon »

sbierwagen wrote: post #78
radicaldreamer wrote: post #56

Not sure what kind of fire there was there, but once those automatic sprinkler systems get going, they are very difficult to stop. Someone in my freshman college dorm decided to use one as a clothes hanger hook and broke the thermometer in there. The sprinkler damaged the entire floor with water and the floor below had spotty rain as well. The fire department came and was mainly concerned about evacuating everyone ra…

Always worth tracking down the sprinkler shut off valve in your residence/place of work. If you’re in a high rise it‘ll be the big red wheel on the sprinkler main in the fire stairs. If it’s a spurious activation you can just shut it off yourself, you don’t need to ask anybody’s permission. The fire department is always going to prioritize safety of life, and after all it’s not their stuff getting soaked.

Please do not disable or tamper with your buildings sprinkler systems.

You will not care about your stuff when you're in jail for negligent manslaughter.

Re: Google Cloud region currently down due to water intrusion

#112

Post by dsfyu404ed »

Kon-Peki wrote: post #106
sbierwagen wrote: post #78

Earlier quoted context omitted.

Always worth tracking down the sprinkler shut off valve in your residence/place of work. If you’re in a high rise it‘ll be the big red wheel on the sprinkler main in the fire stairs. If it’s a spurious activation you can just shut it off yourself, you don’t need to ask anybody’s permission. The fire department is always going to prioritize safety of life, and after all it’s not their stuff getting soaked.

> The fire department is always going to prioritize safety of life, and after all it’s not their stuff getting soaked. They won't hesitate to smash your stuff or break down your walls either. Being in a fire is no joke. You've got to be crazy to think that your stuff is important. It's not.

Anyone talking up firefighters like this clearly hasn't been around them much.

They're boys with toys that they don't frequently get to use and they work for the government. Follow the incentives. They'll do their jobs but they don't give a lot of fucks about things like "unnecessary property damage" and "other people's financial well being" and anything else not written in their KPIs.

I used to drive tow truck. I can't count the number of cars they totaled peeling the roof off (granted some were totaled anyway) because that was easy and cutting a door off was hard. And don't get me started on them and their stupid stands they use to prop shit up in the most questionable of ways...

Re: Google Cloud region currently down due to water intrusion

#113

Post by Eji1700 »

lukax wrote: post #13

Google now has a "data lake" in Paris.

I'm down for coding to go full circle.

We called them bugs because you literally had to go in and get the dead bugs out of your electrical system.

Now we can call it fishing because some pirate has sailed onto your datalake and is looking for sunken hashes.

What do you think the hourly for Cloud Architect/Data lake power boy level 1 should start at?

Re: Google Cloud region currently down due to water intrusion

#114

Post by glogla »

IntelMiner wrote: post #51
lukax wrote: post #13

Google now has a "data lake" in Paris.

Google's DC is underwater OVH's caught fire What's next, us-east-1 gets hit by Godzilla?

Water and fire already had their way, I suspect tornado and a landslide are next.

Re: Google Cloud region currently down due to water intrusion

#116

Post by richardw »

I can imagine clients who used one DC being impacted. But Google’s services would be designed for a single DC going down, right? Data would be eventually consistent (once they find and plug the hard drives in) but isn’t this the promise of the cloud and they’re (approximately) the best at using it.

I have to assume it’s a fault that not even distributed services can paper over. Eg lots of crucial data in flight and they’re reluctant to drop it. Can an expert weigh in?

I love Google’s post-mortems. This one will be epic.

Re: Google Cloud region currently down due to water intrusion

#117

Post by mvanbaak »

radicaldreamer wrote: post #56

Not sure what kind of fire there was there, but once those automatic sprinkler systems get going, they are very difficult to stop. Someone in my freshman college dorm decided to use one as a clothes hanger hook and broke the thermometer in there. The sprinkler damaged the entire floor with water and the floor below had spotty rain as well. The fire department came and was mainly concerned about evacuating everyone ra…

datacenters dont use sprinkler systems (or at least they should not).

Re: Google Cloud region currently down due to water intrusion

#118

Post by local_crmdgeon »

influx wrote: post #91
t0mas88 wrote: post #46

I can't ignore the feeling that Google Cloud is sub par compared to AWS. How did this again cause a multi zone failure. Why haven't they fixed those dependencies the last few times they had a full region failure.

AWS has had datacenters that were flooded, but they failed over and physically moved racks via trucks. Customers never knew.

This has happened much more than people know.

Re: Google Cloud region currently down due to water intrusion

#119

Post by benatkin »

Eji1700 wrote: post #110
zztop44 wrote: post #105

Earlier quoted context omitted.

I don’t see how this is a good argument? Yes, it’s someone else’s computers. That they rent me by the month or the hour or the second through a programmatic interface. That’s exactly the product I want. And it’s not surprising that it has outages sometimes because at the end of the day, yeah, it’s still just a bunch of computers someone has put in a room somewhere.

Yeah this is the usual backlash of experts vs marketing vs management. Someone else managing your shit so you don't have to is a market in just about every industry, and it makes a ton of sense in tech where things don't even have to be on the same continent to work (or very specifically NEED to be on another one if you're international). There's a ton of companies that have jumped to the cloud that probably shouldn'…

No, "serverless" is a toxic term. I get that it exists and I can't do anything about it except refuse to use it. It isn't something I misunderstand because I'm not a marketing person though. It is a term that I understand and disagree with.

Cloud Computing I agree is a usual concept that marketing and experts often disagree about.

Post reply on HN