Live data from Hacker News

Microsoft Azure Outage

twitter.com

211–220 of 247 posts

Re: Microsoft Azure Outage

#211
post #132

Earlier quoted context omitted.

If they were relying on Outlook and Teams to be productive, they probably couldn't really work before either.

What a naive comment. As if the only truly important jobs exist in engineering and require nothing but git and a book on C.

Yeah, what BS. Everyone knows that if you have a book on C, you can always quickly implement git yourself.

Re: Microsoft Azure Outage

#212

Earlier quoted context omitted.

Utter rubbish. Major contracts have account managers and it all gets hashed out 1-1.

Don't know why this was downvoted. We've definitely been able to provide proof of an outage when the status page showed otherwise and get a refund in the form of server credits by contacting them directly. For all 3 big vendors, AWS, Azure, GCP

Agree here as well. It's usually not that hard to provide based on the many, many metrics Azure resources emit that their SLA was breached.

What might be happening is that there is fine print you have to read and be in compliance with in order to be eligible for the SLA.

For example, look at all the conditions which have to be met for a breach of VM SLA in Azure:

https://azure.microsoft.com/en-us/support/legal/sla/virtual-...

Hidden in the SLA details is typically hints on how you can become more resilient in the cloud. So it pays to read the SLA details and really deeply understand what they are telling you.

Re: Microsoft Azure Outage

#213
post #90

Earlier quoted context omitted.

Azure AD is a nightmare. I don't know how many of you sign in to multiple tenants in the console, but it generally involves buying a new computer.

> I don't know how many of you sign in to multiple tenants in the console, but it generally involves buying a new computer. This made me laugh out loud. I'm working in a multi-tenant, multi-subscription environment with Azure AD just now. MS force you to use 2FA and I picked the wrong 2FA app. Now it's completely and utterly comical trying to work out which generated 2FA auth code I need to key in when auth'ing in Vi…

I’m setting up a system with multiple AAD B2C tenants, so I get the joy of switching back and forth between the primary tenant and the B2C tenants frequently (at least until I can finish automating enough of the B2C provisioning bits).

I don’t yet have enough context to fully evaluate against cognito. It may end up being nice to have B2C as a first class AAD tenant, but until I get far enough along to realize those benefits, there will be a lot more cursing under my breath about the need for another layer of identity and the lack of control plane access through azure resource manager APIs/tooling.

Re: Microsoft Azure Outage

#214

Earlier quoted context omitted.

Leave to go where? On-premise and being miserable having to wait months to get a new server with poor automation, observability and worse outages? To another major cloud provider with similar pricing and outages? Cloud helped mostly with automation and scaling but if your system is that critical, you should consider a good CDN as load balancer and multi-cloud (or at least multi-region) for actual robustness.

AWS and GCP both have ~100% uptime in every region for VMs this month. Meanwhile the majority of Azure regions have had various outages in the same period: https://cloudharmony.com/status-of-compute

Almost certainly due to Azure's broken policy where we have critical change advisory's that block deployments for huge periods of time towards the end of the year because of Black Friday and then holidays. Every team has basically been unable to deploy since the week before Thanksgiving when a surprise CCOA was pushed out by leadership at the behest of a certain big customer... then there was the World Cup and the winter holidays. Nobody could really deploy anything from a week before Thanksgiving until a week after the New Years... almost two months worth of batched changes and every team YOLO button pressing as soon as they could in January.

And now layoffs so everyone is super unmotivated! Excellent stuff going on right now from Microsoft senior leadership.

Re: Microsoft Azure Outage

#215

It's good that Microsoft saved money via layoffs so that it balances out when customers leave Azure. Very forward thinking company.

Leave to go where? On-premise and being miserable having to wait months to get a new server with poor automation, observability and worse outages? To another major cloud provider with similar pricing and outages? Cloud helped mostly with automation and scaling but if your system is that critical, you should consider a good CDN as load balancer and multi-cloud (or at least multi-region) for actual robustness.

Where I worked, the internal approval processes and controls over cloud resources are as lengthly as those for on premise hardware. So that may be the case for small companies but I don't think there is much of a difference in those large bureaucracies.

Re: Microsoft Azure Outage

#217
post #202
post #97

Earlier quoted context omitted.

It's mostly Azure though that is badly designed to such an extent that multiple times there have been global outages. In general Azure availability, security (the only major cloud provider with not one but multiple cross-tenant security exploits) and usability are pretty terrible so it shouldn't be used for anything but saying "this is how it should not be done". GCP had a similar thing once, where a BGP update knock…

To be fair, AWS once had a global Route53 outage, which was effectively a global outage for anyone using AWS for DNS.

That outage was limited to Route 53 DNS record editing and not DNS lookups.

Re: Microsoft Azure Outage

#218
post #206
post #202

Earlier quoted context omitted.

To be fair, AWS once had a global Route53 outage, which was effectively a global outage for anyone using AWS for DNS.

Do you have a link to an article about that? My google-fu is weak, and this sounds interesting - that should not happen to DNS - at all - and from the outside Route53 looks quite well managed. So what the heck did they do?

It was back in 2019.

https://twitter.com/AWSSupport/status/1186735657387003904

I forget the details. I do remember half of our internal tools not working at the time due to DNS issues, though. Good times.

Re: Microsoft Azure Outage

#220
Every Azure product I've had to use has been lousy in every possible way. Azure DevOps at my last employer was a nightmare and nobody in the company liked it, not even the managers who decided on it.
Post reply on HN