Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OVH CEO Octave Klaba speaking about the incident [video]

ovh.com

Post reply on HN
Print view View original

91–100 of 143 posts

Re: OVH CEO Octave Klaba speaking about the incident [video]

#91

Post by okl »

bayindirh wrote: post #85
brmgb wrote: post #84

Earlier quoted context omitted.

> oxygen replacing suppressants No, they are a major risk to the employees working there. I would much rather have a data center destroyed by for every twenty years without victims than mandating the user of oxygen replacing fire suppressants.

I understand the concerns you have, however I think there's a good middle ground. Would you consider the following procedure acceptable? - Isolate all rooms with fire-proof doors. - Keep fire supression system at manual. - When fire breaks try to contain (we have 24h watch). - If fails trigger fire supression system. It has 90 second delay and activated per room. - Leave premsises, make the calls. Fire control and su…

"A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools." -- Douglas Adams

Re: OVH CEO Octave Klaba speaking about the incident [video]

#93

Post by ArchOversight »

jacquesm wrote: post #64
yholio wrote: post #50

Earlier quoted context omitted.

The problem with water in the same place with high power equipment is that it instantly turns the room into a death trap for any personnel, now everything is potentially live. Also, in the first part of a lithium battery fire, dropping water on them is quite explosive. It will eventually quench the fire but on the short run it will make it worse, filling the room with explosive hydrogen and poisonous lithium hydroxid…

You would be using Halon injection in a normal DC, after sufficient warning to employees to gtfo if they hadn't done that already.

Halon is not really created anymore. If you have an halon fire suppression system the only way to get recharges is to get it recycled from other systems that are no longer using it.

It's fun when someone accidentally dumps the halon system because they pressed the wrong button, and now halon has to be sourced for replacement.

Re: OVH CEO Octave Klaba speaking about the incident [video]

#94

Post by sschueller »

makkesk8 wrote: post #26

If it turns out it's the upses that caught fire, one can't help but wonder if it would be a better idea to house the upses/backup power solutions in an adjacent smaller building outfitted with sprinklers perhaps?

You would want to use a non-water based fire suppression system.

Re: OVH CEO Octave Klaba speaking about the incident [video]

#95

Post by mattbee »

brmgb wrote: post #84
bayindirh wrote: post #52

Earlier quoted context omitted.

If you're running an IT operation this big, you should have both fire detection units and oxygen replacing suppressants. That should be mandatory. Otherwise, it'd be very hard to contain. Especially when all your servers have RAID controllers with Li-Ion batteries or supercapacitors or other extremely trigger-happy components. Oh, and cooling systems. You're just kindling the fire with it at the beginning.

> oxygen replacing suppressants No, they are a major risk to the employees working there. I would much rather have a data center destroyed by for every twenty years without victims than mandating the user of oxygen replacing fire suppressants.

My old company's DC uses FM200 gas which puts fires out without asphyxiating anyone (I'm assuming the experience would be a respiratory workout for anyone caught in it, but it would have cost £20000 to test).

Re: OVH CEO Octave Klaba speaking about the incident [video]

#96

Post by ArchOversight »

tyingq wrote: post #21
FDSGSG wrote: post #6

Earlier quoted context omitted.

No. OVH has 27(-1) datacenters.

I'm guessing the other two SBG datacenters took some kind of hit. The buildings are very close together: https://cdn.baxtel.com/data-center/ovh-strasbourg-campus/pho... And the fire looked really hot...like melting steel containers hot: https://pbs.twimg.com/media/EwGqV17XMAMF_wa?format=jpg&name=...

SBG 1 had some rooms impacted, SBG 3 and 4 were not impacted at all.

However, they need to bring new power online, as well as bring a new network setup/fiber online to get network connectivity.

Re: OVH CEO Octave Klaba speaking about the incident [video]

#97

Post by Reason077 »

deftnerd wrote: post #23

Are there industry options or methods of wiring to allow for a UPS room separate from the actual rooms the racks are stored in? It's almost tradition to have a rack with UPS's in the bottom and then the rest of the space filled with servers or drive arrays. We wouldn't ever think of putting a tiny backup generator in the bottom of every rack, so why do we put a battery storage system there? Also, with the advances in…

> "it's only a matter of time until Lithium chemistry batteries are available and that also increases the risk of fire."

Not all Li-ion chemistries are equal. In particular, the increasingly popular LiFePO4 (LFP) technology has much higher energy density, longer lifespan, improved environmental characteristics, and similar if not better safety characteristics compared to lead-acid.

(Besides sharing lead-acid's very low risk of fire, LiFePO4 also contains no corrosive acids which could damage and short equipment if a leak were to occur.)

Re: OVH CEO Octave Klaba speaking about the incident [video]

#98

Post by bayindirh »

okl wrote: post #91
bayindirh wrote: post #85

Earlier quoted context omitted.

I understand the concerns you have, however I think there's a good middle ground. Would you consider the following procedure acceptable? - Isolate all rooms with fire-proof doors. - Keep fire supression system at manual. - When fire breaks try to contain (we have 24h watch). - If fails trigger fire supression system. It has 90 second delay and activated per room. - Leave premsises, make the calls. Fire control and su…

"A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools." -- Douglas Adams

"Any attempt to make any system idiot proof will only challenge God to make a better idiot" is the version I like more actually.

Re: OVH CEO Octave Klaba speaking about the incident [video]

#99

Post by bscphil »

nickdothutton wrote: post #20

Lucky it wasn’t a UPS explosion. https://h2tools.org/lessons/battery-room-explosion

I realize it's probably just paranoia on my part but I am terrified of UPSs and can't bring myself to have one in my house. Gigantic batteries lying around in a flammable environment seems way too scary. My always-on server is read-only about 99% of the time, so I just put up with the outages when they happen (about 2 a year). If for some reason my OS eventually gets hosed because of this, I'll rebuild it.

Maybe one day if have a basement and some kind of concrete compartment to put the battery in I'll feel a bit better about it... but even then, not much you can do about gas leakage if that's a possibility with more recent UPSs.

Re: OVH CEO Octave Klaba speaking about the incident [video]

#100

Post by ikiris »

fanf2 wrote: post #75
verytrivial wrote: post #68

Earlier quoted context omitted.

I read his description and literal hand waving as saying that for efficiency reasons, in 2011 they were sort of build like how you'd build a camp fire: convective airflow drawn through and out the top.

Right, this is the OVH “tower” data center design. Here’s a google maps link to the OVH Strasbourg site https://goo.gl/maps/rQjir8byKNwMDZrm9 with (from south to north) SBG1 (a collection of shipping containers), SBG2 (a figure 8 shape), and SBG3 (a box). The idea of the tower design is to take ambient air in through the outside walls, through I think just one row of servers, then into a central void where convection…

AKA a draft furnace.
Post reply on HN