Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fire declared in OVH SBG2 datacentre building

travaux.ovh.net

Post reply on HN
Print view View original

61–70 of 613 posts

Re: Fire declared in OVH SBG2 datacentre building

#61

Post by ericra »

rctay89 wrote: post #2

Puzzles and puzzle storm is down on Lichess: > Due to a fire at one of our data centres, a few of our servers are down and may be down permanently. We are restoring these servers from backups and will enable puzzles and storm as soon as possible. > > We hope that everyone who is dealing with the fire is safe, including the firefighters and everyone at OVH. <3.

I opened two tabs to relax tonight: Hacker News and Lichess. This was the top HN thread, and Lichess is having issues because of the fire.

I didn't know what OVH was before 10 minutes ago, but this seems really impactful. I hope everyone there is safe and that the immediate disaster gets resolved quickly.

Re: Fire declared in OVH SBG2 datacentre building

#62

Post by dylan604 »

AdamJacobMuller wrote: post #54
worldofmatthew wrote: post #11

https://twitter.com/olesovhcom/status/1369504527544705025 "Update 5:20pm. Everybody is safe. Fire destroyed SBG2. A part of SBG1 is destroyed. Firefighters are protecting SBG3. no impact SBG4."

> Fire destroyed SBG2 This is crazy. SBG2 was HUGE and if this isn't a translation error on the part of Octave (which I could understand given the stress and ESL) I have a hard time fathoming what kind of fire could destroy a whole facility with nearly 1000 racks of equipment spread out across separated halls. I'm really hoping "destroyed" here means "we lost all power and network core and there's smoke/fire/physical…

If a fire supression kicks in or the fire department shows up with their hoses, would they still say the fire destroyed it or just say destroyed due to fire and water damage?

Also, fire suppression system do fail. There was an infamous incident in LA for one of the studios. They built a warehouse to be a tape vault with tapes going back to the 80s. A fire started, but the suppression system failed because there was not enough pressure in the system. Total loss. Got to keep your safety equipment properly tested!

Re: Fire declared in OVH SBG2 datacentre building

#64

Post by yongjik »

notaharvardmba wrote: post #48
tothrowaway wrote: post #24

Earlier quoted context omitted.

I'm at OVH as well (in the BHS datacenter, fortunately). I run my entire production system on one beefy machine. The apps and database are replicated to a backup machine hosted with Hetzner (in their Germany datacenter). I also run a tiny VM at OVH which proxies all traffic to Hetzner. I use a failover IP to point at the big rig at OVH. If the main machine fails, I move the failover IP to the VM, which sends all traf…

Bacula has some really cool features for cloud backups. https://bacula.org

Ow what an unfortunate name.

https://en.wikipedia.org/wiki/Baculum

Re: Fire declared in OVH SBG2 datacentre building

#66

Post by proggy »

dylan604 wrote: post #62
AdamJacobMuller wrote: post #54

Earlier quoted context omitted.

> Fire destroyed SBG2 This is crazy. SBG2 was HUGE and if this isn't a translation error on the part of Octave (which I could understand given the stress and ESL) I have a hard time fathoming what kind of fire could destroy a whole facility with nearly 1000 racks of equipment spread out across separated halls. I'm really hoping "destroyed" here means "we lost all power and network core and there's smoke/fire/physical…

If a fire supression kicks in or the fire department shows up with their hoses, would they still say the fire destroyed it or just say destroyed due to fire and water damage? Also, fire suppression system do fail. There was an infamous incident in LA for one of the studios. They built a warehouse to be a tape vault with tapes going back to the 80s. A fire started, but the suppression system failed because there was n…

It wasn’t just “tapes going back to the 80s.” Those were just the media Universal initially admitted to losing. No, that building was the mother lode. It had film archives going back over 50 years, and worst of all — unreleased audio tape masters for thousands upon thousands of recording artists. The amount of “remastered” album potential that fire destroyed is probably in the billions of dollars, let alone the historical loss of all those recordings by historical persons that will never be heard due to a failure of a fire prevention system. Fascinating case study in why you should never put all your eggs in one basket.

https://en.m.wikipedia.org/wiki/2008_Universal_Studios_fire

Re: Fire declared in OVH SBG2 datacentre building

#67

Post by Shank »

A status update on the OVH tracker for a different datacenter (LIM-1 / Limburg) says "We are going to intervene in the rack to replace a large number of power supply cables that could have an insulation defect." [0][1] The same type of issue is "planned" in BHS [3] and GRA [2].

Eerie timing: do they possibly suspect some bad cables?

[0]: http://travaux.ovh.net/?do=details&id=49016

[1]: http://travaux.ovh.net/?do=details&id=49017

[3]: http://travaux.ovh.net/?do=details&id=49462

[2]: http://travaux.ovh.net/?do=details&id=49465

Re: Fire declared in OVH SBG2 datacentre building

#70

Post by neo34 »

"Everyone is safe. Fire has destroyed SBG2. A part of SBG1 is destroyed. The firefighters are protecting SBG3. No impact on SBG4". Tweet from Octave Klaba, founder of OVHcloud. "All our clients on this site are possibly impacted"

Tout le monde est sain et sauf. Le feu a détruit SBG2. Une partie de SBG1 est détruite. Les pompiers protègent actuellement SBG3. Pas d’impact sur SBG4 », a tweeté Octave Klaba, le fondateur d’OVHcloud, en désignant les différentes parties du site. « Tous nos clients sur ce centre de données sont susceptibles d’être impactés », a précisé l’entreprise sur Twitter.

Post reply on HN