Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OVH Incident in Strasbourg

status.ovh.com

Post reply on HN
Print view View original

121–130 of 207 posts

Re: OVH Incident in Strasbourg

#121

Post by Sami_Lehtinen »

tmikaeld wrote: post #114
Sami_Lehtinen wrote: post #111

Earlier quoted context omitted.

We're nerds, so how about checking the facts (?). Please explain me if there has been some kind of time anomaly lately or? My submit timestamp: 07:21:25 Your submit timestamp: 07:28:23 About the title. I did consider the title for a while, because I wasn't sure how bad the situation was. But from my own independent monitoring system I did see that RBX and SBG servers were unavailable. Of course I also did some basic…

Sorry, I didn't know how to check the timestamps. Yeah, I expected this to clear up in a matter of minutes. Now it seems to be a shitstorm of historic proportions...

I'm waiting for the postmortem. I'm very curious to see, what was the root cause of all this mess. As usual, there probably were several overlapping causes.

Re: OVH Incident in Strasbourg

#122

Post by smoyer »

Cthulhu_ wrote: post #118
wiz21c wrote: post #104

Damn, every emergency power supply I have encountered (the big ones with fuel and hundreds of batteries) always fail to start when they have to... Why is that ?

Lack of testing; ideally you throw a switch once a month or so to make sure the UPSes work and the generators power on, but I can imagine it's something you'd rather not do, if it doesn't work you've just intentionally borked things for your customers. Still, it is something that datacenters should do, do it a few times before you have any customers, keep doing it. Maybe offer a discount (if it's a new datacenter for…

We always had our generators cycle on once a week for 20 minutes or so without assuming load. You don't want the (in our case propane) engines sitting too long without being run.

Re: OVH Incident in Strasbourg

#123

Post by vabene1111 »

tyingq wrote: post #109
vabene1111 wrote: post #52

Earlier quoted context omitted.

its OVH: The Hardware is good DDoS protection is good The Prices are high but support/administration does not work well, i have a lot of really weird story's with them, from them plugging in a keyboard in our server to reboot it (without any reason) to taking down a server for a requested maintenance only to notice after 4 hours of downtime that they did not ask their bosses if they were allowed to even perform the m…

>ts OVH: The Hardware is good DDoS protection is good The Prices are high The prices are high? Compared to what? Cheap is their raison d'être.

compared to other dedicated Server Hardware, not talking about business Cloud Infrastructure, no idea about that.

Sry if that caused confusion

Re: OVH Incident in Strasbourg

#124

Post by Filligree »

tyingq wrote: post #109
vabene1111 wrote: post #52

Earlier quoted context omitted.

its OVH: The Hardware is good DDoS protection is good The Prices are high but support/administration does not work well, i have a lot of really weird story's with them, from them plugging in a keyboard in our server to reboot it (without any reason) to taking down a server for a requested maintenance only to notice after 4 hours of downtime that they did not ask their bosses if they were allowed to even perform the m…

>ts OVH: The Hardware is good DDoS protection is good The Prices are high The prices are high? Compared to what? Cheap is their raison d'être.

Well, Hetzner for example.

Re: OVH Incident in Strasbourg

#125

Post by pmontra »

The status page is up again http://status.ovh.net/

I paste the report so far:

-------------

FS#15162 — SBG

Attached to Project— Network

Task Type: Incident

Category: Strasbourg

Status: In progress

Percent Complete: 0%

Details

We are experiencing an electrical outage on Strasbourg site.

We are investigating.

Comments (2)

Comment by OVH - Thursday, 09 November 2017, 10:55AM

SBG: ERDF repared 1 line 20KV. the second is still down. All Gens are UP. 2 routing rooms coming UP. SBG2 will be UP in 15-20min (boot time). SBG1/SBG4: 1h-2h

Comment by OVH - Thursday, 09 November 2017, 12:04PM

Traffic is getting back up. About 30% of the IP are now UP and running.

-------------

VPSes are still marked as read in the dashboard. I can't access mine.

Re: OVH Incident in Strasbourg

#127

Post by oelmekki »

Btw, note for those who use ovh ISP like me (this is a thing in France): your connection works, only the DNS's do not.

Fix (debian-like):

    sudo apt-get install bind9
Then put in /etc/resolv.conf, if it's not already there:

    nameserver 127.0.1.1
This runs a local nameserver that you use directly for resolving.

Oh, obviously, you need resolving to install the resolver :) Hope you have a 4g connection available.

Alternatively, you can just use google dns:

    nameserver 8.8.8.8
    nameserver 8.8.4.4

Re: OVH Incident in Strasbourg

#129

Post by dspillett »

jakub_g wrote: post #100
roblabla wrote: post #87

Earlier quoted context omitted.

The status page was down during the outage.

If so, then it's just like Amazon's status page during the AWS outage [1]. Pro-tip: self-hosting status page is maybe not the best idea. [1] https://twitter.com/awscloud/status/836656664635846656?lang=...

Like the old Red Dwarf episode:

Lister: What's the damage Hol?

Holly: I don't know Dave. The damage report machine has been damaged.

Re: OVH Incident in Strasbourg

#130

Post by [deleted] »

oelmekki wrote: post #127

Btw, note for those who use ovh ISP like me (this is a thing in France): your connection works, only the DNS's do not. Fix (debian-like): sudo apt-get install bind9 Then put in /etc/resolv.conf, if it's not already there: nameserver 127.0.1.1 This runs a local nameserver that you use directly for resolving. Oh, obviously, you need resolving to install the resolver :) Hope you have a 4g connection available. Alternati…

[deleted]
Post reply on HN