Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fire declared in OVH SBG2 datacentre building

travaux.ovh.net

Post reply on HN
Print view View original

381–390 of 613 posts

Re: Fire declared in OVH SBG2 datacentre building

#381

Post by pmlnr »

clan wrote: post #332
pmlnr wrote: post #294

Earlier quoted context omitted.

I only have a personal server running in Hetzner but it's mirrored onto a tiny local computer at home. They both run postfix + dovecot, so mail is synced via dovecot replication. Data is rsync-ed daily, and everything has ZFS snapshots. MySQL is not set into replication - my home internet breaks often enough to have serious issues, so instead I drop everything every day import a full dump from the main server, and do…

Not saying that you should never do a full mysql dump. Nor that you should not ensure that you can import a full dump. But when you already use ZFS you can do a very speedy full backup with: mysql Transfer the snapshot off-site (and test!). Either as a simple filecopy (the snapshot ensured a consistent database) or a little more advanced with zfs send/receive. This is much quicker and more painless than mysql dump. E…

Good point, but my DB is tiny, so for now, I can afford the mysqldump. But I'll keep this in mind.

Re: Fire declared in OVH SBG2 datacentre building

#383

Post by lovedswain »

> In the case of Roubaix 4, the Datacenter is made with a lot of wood:

> Finally, we have other photos of the floor of the OVH "Roubaix 4" tower. It is clearly wood! Hope it's fireproof wood! A wooden datacenter ... is still original, we must admit.

> In France, data centers are mainly regulated by the labor code, by ICPE recommendations (with authorization or declaration) and by insurers. At the purely regulatory level, the only things that are required are:

> - Mechanical or natural smoke extraction for blind premises or those covering more than 300m2

> - The fire compartmentalization beyond a certain volume / m2

> - Emergency exits accessible with a certain width

> - Ventilation giving a minimum of fresh air per occupant

> - Access to the firefighter from the facade for the premises are the low floor of the last level is more than 8 meters

> - 1 toilet for 10 people (occupying a position considered "fixed")

https://lafibre.info/ovh-datacenter/ovh-et-la-protection-inc...

Re: Fire declared in OVH SBG2 datacentre building

#384

Post by yread »

atkbrah wrote: post #71
qbasic_forever wrote: post #60

Earlier quoted context omitted.

Just guessing, but maybe a fire suppression system going off could wipe out all the machines? The couple datacenters I've been inside were small, old and used halon gas which wasn't supposed to destroy the machines. No idea how it works in big places these days.

A few years back there was an incident in Sweden where noise coming from the gas based fire suppression system going off destroyed hard drives [1]. 1. https://www.theregister.com/2018/04/26/decibels_destroy_disk...

We had the same issue at a customer site. To add since the decibels were outside the rated environment the warranty was void on the harddisks and they had to be replaced even if they were not destroyed.

Re: Fire declared in OVH SBG2 datacentre building

#385

Post by jacquesm »

tomatocracy wrote: post #290
jacquesm wrote: post #150

Earlier quoted context omitted.

The final layer is call the insurance company.

There are always uninsurable events and for large enough companies/risks there are also liquidity limits to the size of coverage you can get from the market even for insurable events. As such, it makes sense to make the level of risk you plan to accept (by not being insured against it and not mitigating) a conscious economic decision rather than pretending you've covered everything.

As long as you have outside shareholders you can decide that. If you do you'd be surprised about how they will respond to an attitude like that. After all: you can decide the levels of risk that you personally are comfortable with leading to extinguishing of the business, but a typical shareholder is looking at you to protect their investment and not insuring against a known risk which at some point in time materializes is an excellent way to find yourself in the crosshairs of a minority shareholder lawsuit against a (former) company executive.

Re: Fire declared in OVH SBG2 datacentre building

#386

Post by jacquesm »

corty wrote: post #280
jacquesm wrote: post #150

Earlier quoted context omitted.

The final layer is call the insurance company.

Not really, the insurance won't make things right in an instant. They will usually compensate you financially, but often only after painstaking evaluation of all circumstances, weighing their chances in court to get out of paying you and maybe a lengthy court battle and a race against your bankruptcy. So yes, getting insurance can be a good idea to offset some losses you may have, as long as they are somewhat limited…

Fair enough.

Re: Fire declared in OVH SBG2 datacentre building

#387

Post by leajkinUnk »

exikyut wrote: post #325
leajkinUnk wrote: post #36

Earlier quoted context omitted.

“Big cloud” has had fires take out clusters, and somehow they manage to keep it out of the news. In spite of the redundancy and failover procedures, keeping your data centers running when one of the clusters was recently *on fire* is something that is often only possible due to heroic efforts. When I say “heroic efforts”, that’s in contrast to “ordinary error recovery and failover”, which is the way you’d want to han…

I'm curious what references or leads I might follow to learn more about these fires and other events you mention.

Get a job working at these companies and go out for drinks with the old-timers.

Re: Fire declared in OVH SBG2 datacentre building

#388

Post by whalesalad »

aetherspawn wrote: post #167
nikanj wrote: post #146

According to the official status page, the whole datacenter is still green http://status.ovh.com/vms/index_sbg2.html

You're completely right. IMO this is the best comment in this whole thread. Their status page must be broken, or it's a lie.

lol... that’s how most status pages are

Re: Fire declared in OVH SBG2 datacentre building

#390

Post by klingon78 »

buran77 wrote: post #377
klingon78 wrote: post #320

Earlier quoted context omitted.

I have a reliability and risk avoidance mindset, but I’ve had to stand back because my mental gas tank for trying to keep things going is near empty. I’ve really struggled working with others that either are both ignorant and apathetic about the business’s ability to deal with risk or believe that it’s their job to keep putting duct tape over the duct tape that breaks multiple times a day while users struggle. I like…

Reliability is there to keep your things running smoothly during normal operations. Backups are there for when you reach the end of your reliability rope. Neither is really a good replacement for the other. The most reliable systems will still fail eventually, and the best of backups can't run your day to day operations. At the end of the day you have a budget (of any kind) and a list of priorities on which to spend…

Backups are a reliability tool, yes.

A backup on its own is of little worth if unused.

When a backup is used to re-enable something, then the amount of time disabled may be decreased. When it is, this is reliability- we keep things usable and in function, more than not.

Post reply on HN