Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fire declared in OVH SBG2 datacentre building

travaux.ovh.net

Post reply on HN
Print view View original

391–400 of 613 posts

Re: Fire declared in OVH SBG2 datacentre building

#391

Post by pcl »

sparrish wrote: post #38
geocrasher wrote: post #10

The classic "lp0 on fire" error message comes to mind: https://en.wikipedia.org/wiki/Lp0_on_fire Really though, I feel truly awful for anyone affected by this. The post recommends implementing a disaster recovery plan. The truth is that most people don't have one. So, let's use this post to talk about Disaster Recovery Plans! Mine: I have 5 servers at OVH (not at SBG) and they all back up to Amazon S3 or Backblaze B2…

Got burned once (no pun intended), learned my lesson. Hot spare on a different continent with replicated data along with a third box just for backups. The backup box gets offsite backups held in a safe with another redundant copy in another site in another safe. Restores are tested quarterly. Keep backups of backups. Once bitten, twice shy.

> Restores are tested quarterly.

Probably this is the most important part of your plan. It's not the backup that matters; it's the restore. And if you don't practice it from time to time, it's probably not going to work when you need it.

Re: Fire declared in OVH SBG2 datacentre building

#392

Post by malobre »

lovedswain wrote: post #383

> In the case of Roubaix 4, the Datacenter is made with a lot of wood: > Finally, we have other photos of the floor of the OVH "Roubaix 4" tower. It is clearly wood! Hope it's fireproof wood! A wooden datacenter ... is still original, we must admit. > In France, data centers are mainly regulated by the labor code, by ICPE recommendations (with authorization or declaration) and by insurers. At the purely regulatory le…

"1 chiotte" -> "1 toilet" not puppy

Re: Fire declared in OVH SBG2 datacentre building

#393

Post by jaywalk »

dboreham wrote: post #367
SergeAx wrote: post #366

Earlier quoted context omitted.

I don't have to "backup servers" for a long time now. I have an Ansible playbook to deploy and orchestrate services, which, in turn, are mostly dockerized. So my recovery plan is to turn on "sorry, maintenance" banner via CDN, spin up a bunch of new VPSes, run Ansible scenario for deployment and restore database from hidden replica or latest dump.

> restore database from hidden replica or latest dump You do have backup servers.

He said he doesn't have to backup servers, not that he doesn't have backup servers.

Re: Fire declared in OVH SBG2 datacentre building

#394

Post by ricardobeat »

> At 00:47 on Wednesday, March 10, 2021, a fire broke out in a room in one of our 4 datacenters in Strasbourg, SBG2. Please note that the site is not classified as a Seveso site.

> Firefighters immediately intervened to protect our teams and prevent the spread of the fire. At 2:54 am they isolated the site and closed off its perimeter.

> By 4:09 am, the fire had destroyed SBG2 and continued to present risks to the nearby datacenters until the fire brigade brought the fire under control.

> From 5:30 am, the site has been unavailable to our teams for obvious security reasons, under the direction of the prefecture. The fire is now contained.

Re: Fire declared in OVH SBG2 datacentre building

#395

Post by dolmen »

lebaux wrote: post #333
jakub_g wrote: post #305

Earlier quoted context omitted.

Also, 3 years ago they had an outage in Strasbourg, and the status page was down apparently as a result of the outage. https://news.ycombinator.com/item?id=15661218 They are not the only ones though. All too common. Well, it's tricky to set this up properly. The only proper way would be to use external infra for the status page.

Isn’t that why things like statuspage.io exist, though?

Where is statuspage.io hosted?

Re: Fire declared in OVH SBG2 datacentre building

#397

Post by edejong »

saagarjha wrote: post #128
jhugo wrote: post #110

Earlier quoted context omitted.

At a datacentre I used to visit years ago, part of the site induction was learning that if the fire suppression alarm went off, you had a certain amount of time to get out of the room before the argon would deploy, so you should always have a path to the nearest exit in mind. The implication was that it wasn't safe to be in the room once it deployed, but I don't know for sure.

Presumably you would have trouble breathing as it displaced the oxygen in the room?

I heard that the pressure difference might rupture your ear drums.

Re: Fire declared in OVH SBG2 datacentre building

#398

Post by eloff »

omnimus wrote: post #378
7ewis wrote: post #356

Earlier quoted context omitted.

Not done any research into it, but I always thought OVH was supposed to be a very budget VPS service primarily for personal use rather than business. Although thought it was akin to having a Raspberry Pi plugged in at home. Again, I may be completely wrong but why would you not use AWS/GCP? Even if it's complexity, Amazon have Lightsail, or if it's cost I thought DigitalOcean was one of the only reputable business-gr…

You cant imagine it yet big chunk of the independent internet runs on small vps servers. There isnt much difference between DO and OVH, Hetzner, Vultr, Linode... not sure why DO would be better. I mean its US company doing marketing right. Thats the difference. Plus ovh/hetzner have only EU locations. I think small bussinesses like smaller simple providers instead of bigclouds. Its different philosophy if you are afr…

I can think of a lot of big differences. For one you can get much larger machines at OVH and Hetzner with fancy storage configurations for your database if desired (e.g. Optane for your indices, magnetic drives for your transaction log, and raided SSDs for the tables)

They also don't charge for bandwidth, although some of those other providers have a generous free bandwidth and cheap overage.

Re: Fire declared in OVH SBG2 datacentre building

#399

Post by bleuarff »

omnimus wrote: post #378
7ewis wrote: post #356

Earlier quoted context omitted.

Not done any research into it, but I always thought OVH was supposed to be a very budget VPS service primarily for personal use rather than business. Although thought it was akin to having a Raspberry Pi plugged in at home. Again, I may be completely wrong but why would you not use AWS/GCP? Even if it's complexity, Amazon have Lightsail, or if it's cost I thought DigitalOcean was one of the only reputable business-gr…

You cant imagine it yet big chunk of the independent internet runs on small vps servers. There isnt much difference between DO and OVH, Hetzner, Vultr, Linode... not sure why DO would be better. I mean its US company doing marketing right. Thats the difference. Plus ovh/hetzner have only EU locations. I think small bussinesses like smaller simple providers instead of bigclouds. Its different philosophy if you are afr…

OVH is not Europe-only, it has datacenters in America, Asia and Australia[1].

[1] https://www.ovh.com/world/us/about-us/datacenters.xml

Re: Fire declared in OVH SBG2 datacentre building

#400

Post by thspimpolds »

jhugo wrote: post #220
saagarjha wrote: post #128

Earlier quoted context omitted.

Presumably you would have trouble breathing as it displaced the oxygen in the room?

Yeah, that was always my understanding. GP saying "the gas is designed to be used in rooms with people, so that [it] can be triggered any time" made me second-guess that though. Maybe there is a concentration of oxygen that is high enough for humans to survive, yet too low to sustain combustion?

The "gas" is more like a fog of capsules. FM-200 is a common one. Basically it has a Fire suppression agent inside crystals which are blasted into the room by compressed air. These crystals melt when they get over a certain temperature and therefore won't kill you; however, breathing that in isn't really pleasant.

Source: I've been in an FM-200 discharge

Post reply on HN