OVH CEO Octave Klaba speaking about the incident [video]

Post by dilyevsky » Fri, Mar 12, 2021, 4:31 AM UTC

mikepurvis wrote: post #34

deftnerd wrote: post #23

Are there industry options or methods of wiring to allow for a UPS room separate from the actual rooms the racks are stored in? It's almost tradition to have a rack with UPS's in the bottom and then the rest of the space filled with servers or drive arrays. We wouldn't ever think of putting a tiny backup generator in the bottom of every rack, so why do we put a battery storage system there? Also, with the advances in…

For the extreme opposite of that, Google famously trolled everyone 10 years ago by announcing that every one of their servers had its own in-chassis 12V battery: https://www.cnet.com/news/google-uncloaks-once-secret-server...

Post by tmikaeld » Fri, Mar 12, 2021, 5:06 AM UTC

ev1 wrote: post #59

kuschku wrote: post #58

Earlier quoted context omitted.

2 OVH dedicated servers in different countries are still cheaper than one AWS instance. e.g. I've got servers at OVH SBG-2, Hetzner's Falckenstein, and Online.net's AMS datacenters — the total of which is still almost a magnitude less than the same cost on AWS or GCP (granted, that's including traffic)

Pretty much. Use 5% of the money you saved moving your workload from AWS to OVH to support failover, DR, and backups. You can probably buy like five or six machines of equivalent spec or more.

Post by bayindirh » Fri, Mar 12, 2021, 8:26 AM UTC

brmgb wrote: post #84

bayindirh wrote: post #52

Earlier quoted context omitted.

If you're running an IT operation this big, you should have both fire detection units and oxygen replacing suppressants. That should be mandatory. Otherwise, it'd be very hard to contain. Especially when all your servers have RAID controllers with Li-Ion batteries or supercapacitors or other extremely trigger-happy components. Oh, and cooling systems. You're just kindling the fire with it at the beginning.

> oxygen replacing suppressants No, they are a major risk to the employees working there. I would much rather have a data center destroyed by for every twenty years without victims than mandating the user of oxygen replacing fire suppressants.

    - Isolate all rooms with fire-proof doors.
    - Keep fire supression system at manual.
    - When fire breaks try to contain (we have 24h watch).
    - If fails trigger fire supression system. It has 90 second delay and activated per room.
    - Leave premsises, make the calls.

Post by londons_explore » Fri, Mar 12, 2021, 8:43 AM UTC

dilyevsky wrote: post #82

Saris wrote: post #33

Earlier quoted context omitted.

It also depends if you're renting a dedicated server, vs cloud/VPS. AWS/Google/Azure deal with virtualized systems that can be moved around to another server easily. OVH has a lot of dedicated servers as well though, so if you're using one of those then it can't be moved very easily to avoid downtime.

Just to note - only google does live migration (with ~100ms blackout) with others you will take some downtime anyway (assuming the remaining zones even have enough capacity to fit everyone)

Post by pcl » Fri, Mar 12, 2021, 9:26 AM UTC

bayindirh wrote: post #85

brmgb wrote: post #84

Earlier quoted context omitted.

> oxygen replacing suppressants No, they are a major risk to the employees working there. I would much rather have a data center destroyed by for every twenty years without victims than mandating the user of oxygen replacing fire suppressants.

I understand the concerns you have, however I think there's a good middle ground. Would you consider the following procedure acceptable? - Isolate all rooms with fire-proof doors. - Keep fire supression system at manual. - When fire breaks try to contain (we have 24h watch). - If fails trigger fire supression system. It has 90 second delay and activated per room. - Leave premsises, make the calls. Fire control and su…