Live data from Hacker News

Microsoft Azure suffers outage after cooling issue

datacenterdynamics.com

71–80 of 113 posts

Re: Microsoft Azure suffers outage after cooling issue

#72

Earlier quoted context omitted.

In my experience git is excellent at being offline. I’m not sure if we would have had problem if our team foundation setup had ever went off line, but we moved to git long ago because it’s what every single IT-based student learns in my country. With every potential hire knowing git and almost none of them knowing tfd/vsts, it seemed silly not to migrate since the platforms both work well for what we need and git was…

In my experience, most companies that use "github" (and not "git") cannot get much done if github is down.

So much this. All the pipeline and delivery stuff is glued into github’s API. Most companies aren’t using git any more but github and git is just a dependency. I’ve even seen developers who are unaware that git can exist without github.

I incidentally think this is a completely terrible outcome.

Re: Microsoft Azure suffers outage after cooling issue

#73

Earlier quoted context omitted.

In my experience git is excellent at being offline. I’m not sure if we would have had problem if our team foundation setup had ever went off line, but we moved to git long ago because it’s what every single IT-based student learns in my country. With every potential hire knowing git and almost none of them knowing tfd/vsts, it seemed silly not to migrate since the platforms both work well for what we need and git was…

In my experience, most companies that use "github" (and not "git") cannot get much done if github is down.

Or badly configured CI, that redownloads dependencies all the time instead of caching them locally.

Re: Microsoft Azure suffers outage after cooling issue

#74

Earlier quoted context omitted.

In my experience git is excellent at being offline. I’m not sure if we would have had problem if our team foundation setup had ever went off line, but we moved to git long ago because it’s what every single IT-based student learns in my country. With every potential hire knowing git and almost none of them knowing tfd/vsts, it seemed silly not to migrate since the platforms both work well for what we need and git was…

In my experience, most companies that use "github" (and not "git") cannot get much done if github is down.

It doesn't really mater what service provider you use. You can't do anything that requires a sync if the server is down. You can do a bit of local development, but things like automated tests and deployment to staging servers will tend to break if they can't access the source.

Re: Microsoft Azure suffers outage after cooling issue

#75

The Visual Studio Marketplace is also down https://marketplace.visualstudio.com/ .

Just today I was having issues with the Prettier extension in VS Code, and I uninstalled it to see if that would fix it (I read that usually fixes the issues I was having). Then I realized that I couldn't install it again because VS Marketplace was down. This was like 8 hours ago and still no signs of recovery. Of course, all my builds are failing because of some stupid formatting issue that Prettier usually would so…

> Of course, all my builds are failing because of some stupid formatting issue that Prettier usually would solve, so yeah..thanks MSFT.

Is failing builds due to formatting issues really a sound setup?

Re: Microsoft Azure suffers outage after cooling issue

#76
post #53

Earlier quoted context omitted.

Exactly, this happens even in just a normal production failure. I don't know what else they could have said/communicated. Not to mention this is the 7th largest data center in the world, resolving the problem likely took/is taking a long long time just because there are so many machines. I was lucky that the only outage effect I've suffered from is that my storage is locked, which means I can't add new file/edit code…

Were you able to perform operations given ARM services were down? Were you using your secondary DB for read-only or did you fail-over some other way?

How often do we do this for our customers? Honestly, if a customer wanted to know that much detail about my process, I would be quite bothered, and I suspect most people would be as well. Wait for the recap, everything will become clear then. You don't need to know everything in the moment.

Re: Microsoft Azure suffers outage after cooling issue

#77

Earlier quoted context omitted.

I'm sure services I use run on all of the major cloud providers, but if that page was supposed to be enlightening, I only recognized one brand from the first page of customers. I refreshed a couple times, and sure, I saw more (on average 1 or 2) that I recognized on each page. But I don't think your response is particularly persuasive. Are you suggesting that the services that I use that run on AWS are in fact, not s…

>I'm sure services I use run on all of the major cloud providers, but if that page was supposed to be enlightening, I only recognized one brand from the first page of customers. So the first page was supposed to be indicative of all of the popular consumer facing services they host? Here, let me help you out: Spotify, eBay, Twitter, Apple iCloud, Verizon, Vimeo, Netflix, etc >I refreshed a couple times, and sure, I s…

We've banned this account. All it has done is aggressively post pro-Google comments and diss Google competitors.

Single-purpose accounts are not allowed here, especially not when pushing an agenda, and most of all not when pushing corporate propaganda. Of all the things that make HN users angry, that's at the top. And I agree with them.

Most of the time we tell HN users that they're not allowed to accuse each other of astroturfing. When we do find a clear-cut case of abuse that's been getting away with it for this long, I get pretty steamed.

You've also frequently broken the site guidelines by being uncivil, so much so that we've warned you at least half a dozen times. That's more than enough reason to ban you in its own right.

Re: Microsoft Azure suffers outage after cooling issue

#78

Earlier quoted context omitted.

The biggest and most popular services run on Google Cloud. https://cloud.google.com/customers/ You know....services consumers actually use.

I'm sure services I use run on all of the major cloud providers, but if that page was supposed to be enlightening, I only recognized one brand from the first page of customers. I refreshed a couple times, and sure, I saw more (on average 1 or 2) that I recognized on each page. But I don't think your response is particularly persuasive. Are you suggesting that the services that I use that run on AWS are in fact, not s…

In the future, if you or anyone runs across an account that has been abusing HN in this way, it would be much better to email us at hn@ycombinator.com so we can look into it. I happened to randomly see your comment, but the odds of that are low.

Re: Microsoft Azure suffers outage after cooling issue

#79
post #53

Earlier quoted context omitted.

Were you able to perform operations given ARM services were down? Were you using your secondary DB for read-only or did you fail-over some other way?

How often do we do this for our customers? Honestly, if a customer wanted to know that much detail about my process, I would be quite bothered, and I suspect most people would be as well. Wait for the recap, everything will become clear then. You don't need to know everything in the moment.

I'm not asking for every detail. It's horrible to wait 20 hours and counting for unknown reasons and to not have something to work towards. I only ask for more transparency is all.

It's akin to waiting for surgery and the doctor saying "we're working on it". I don't want/need the details for the surgery, but tell me everything is ok, what comes next and give me some estimates to set expectations.

Re: Microsoft Azure suffers outage after cooling issue

#80
post #41

Here are some official links on the issue: - https://azure.microsoft.com/en-us/status/ - https://twitter.com/AzureSupport - https://twitter.com/Office365Status - https://status.office.com

As of 07:15 UTC on the 5th of September 2018 the Azure status message reported:

"NEXT UPDATE: The next update will be provided by 07:00 UTC 05 Sep 2018 or as events warrant."

As I finished writing this they finally updated with essentially the same message except stalling for an additional two hours.

So, if you're thinking "Well, surprises will happen". Yeah, and Microsoft is not actually prepared for that at all, so, sucks to be their customer I guess?

Post reply on HN