Live data from Hacker News

Microsoft Azure Outage

azure.microsoft.com

51–60 of 178 posts

Re: Microsoft Azure Outage

#51
post #34

Earlier quoted context omitted.

Can you point to an example of this happening on AWS in multiple regions simultaneously?

Yes. The day half the internet seemed to be down. It took Amazon more than 24 hours to recover, and having your services in multiple availability zones did not shield you from the failure. http://aws.amazon.com/message/65648/

This seems to have only affected one region. Am I missing something?

Re: Microsoft Azure Outage

#52

Earlier quoted context omitted.

Mine have been gradually coming back, too. Timing couldn't have been any better for me. Some Alanis material there: https://twitter.com/bizspark/status/534858596748906496 Now I'll have to distribute between AWS and Azure, too.

They all have outages periodically, in my experience.

Yeah, that's the main thing to take away from these events. I've never seen a system (cloud, local, co-located, or otherwise) with 100% uptime, despite every effort to the contrary. Even sites like Facebook and Google have downtime. In the last few years that I've been using it, Azure has been at least as stable as other cloud providers.

That doesn't help with the awful timing though. Ouch. I just Buffer-retweeted your BizSpark tweet above, scheduled for tomorrow. Maybe a little bump now that you're back up and running will help ease the pain...

Re: Microsoft Azure Outage

#53
post #49
post #43

Earlier quoted context omitted.

Major outages should absolutely weigh into your decisions as to what platform to use. That being said you can mitigate the effect of instability by engineering your app to failover to other availability zones or even to another cloud platform (depending on your app) if the entire platform goes down. Obviously there is a segnifigant cost associated with engineering this level of cross platform redundancy which is why…

Thanks! This is very helpful. Our big hosted site is hosted in Europe is actually working but our blogs and a news website are both down. We offer a paid service at 600$ a year and if the main site was down it would be very bad for our reputation. Our DNS points to Azure on all these domains and things are hosted as "Azure Web Site" - how would notifications work if Azure itself is failing? Would I need to proxy the…

There are any number of uptime, and ping services that you can google for. This can raise the alarm in a timely fashion when your site, or parts of your site go down, and then you decide how to handle those issues.

You may also want to google for DNS failover services, to help you automatically redirect traffic in more catastrophic failure cases. There are offerings from google[1], AWS[2], and others.

[1]: https://cloud.google.com/dns/docs

[2]: http://aws.amazon.com/route53/

Re: Microsoft Azure Outage

#54
post #51

Earlier quoted context omitted.

Yes. The day half the internet seemed to be down. It took Amazon more than 24 hours to recover, and having your services in multiple availability zones did not shield you from the failure. http://aws.amazon.com/message/65648/

This seems to have only affected one region. Am I missing something?

Yes. It started as a failure in one region, and propagated to others as it overloaded the "control plane" -- the stuff that runs "the cloud", and EBS tried to replicate "failed" disks to the point that Amazon ran out of disk space in the cluster. At the time, I was paying for RDS Multi-AZ which runs your database in multiple availability zones at once with hot failover if the primary goes offline. It failed to fail over despite that. Many large sites went down for a very long time that day, and people couldn't spawn replacement instances even in other AZs than the one the failure started in.

Re: Microsoft Azure Outage

#56
post #51

Earlier quoted context omitted.

Yes. The day half the internet seemed to be down. It took Amazon more than 24 hours to recover, and having your services in multiple availability zones did not shield you from the failure. http://aws.amazon.com/message/65648/

This seems to have only affected one region. Am I missing something?

It was one region, multiple availability zones. You're right Multi-AZ != Multi-region (for things like Sandy and natural disasters) BUT they have mostly separate infra which does make multi-az failures very unlikely. Due to its simplicity (with VPC and stuff) some people (perhaps wrongly) treat multi-az like multi-region.

Re: Microsoft Azure Outage

#57
post #38

This outage exposes the clowns that actually chose Azure as their cloud provider. If you use AMZN and it goes down, at least you're in good company, with the likes of Netflix, Twitter, Instagram, and so on. It's like yeah, I'm big like they are. So what, it went down, so is Netflix. What does your client/customer think of you being on Azure? That you chose the crappy solution because your low-tech infrastructure stil…

Over 80% of the Fortune 500 run on Azure. 20% of Azure VMs are Linux. You are not well informed.

"run on" - I suspect you're being fed a unicode pile of poo here.

More likely the have _something_ which runs on Azure. Fortune 500s are, pretty much by definition, quite large - and probably have tons of departments and sub departments. And at least one of those departments probably has a task of trying out new things, like Azure, by running something on it.

What surprises me is that nearly 20% of Fortune 500s _don't_ have something running on Azure.

(I wonder what percentage "run on" Amazon)

Re: Microsoft Azure Outage

#59
post #51

Earlier quoted context omitted.

This seems to have only affected one region. Am I missing something?

Yes. It started as a failure in one region, and propagated to others as it overloaded the "control plane" -- the stuff that runs "the cloud", and EBS tried to replicate "failed" disks to the point that Amazon ran out of disk space in the cluster. At the time, I was paying for RDS Multi-AZ which runs your database in multiple availability zones at once with hot failover if the primary goes offline. It failed to fail o…

You're confusing region with AZ. They've never had a multi-region outage (yet).

Re: Microsoft Azure Outage

#60

I feel like an idiot. MS featured my Azure startup today, quoting me about overall stability etc (which has been the case for us, until today). They then proceeded to go down, taking all our production systems with them. (yes we do have AWS, too) Sigh.

Azure has had quite a few outages this year. I'd say it's already lower than that 99.999 percent uptime or w/e they are advertising.
Post reply on HN