Live data from Hacker News

Tell HN: Azure outage

news.ycombinator.com

241–250 of 841 posts

Re: Tell HN: Azure outage

#241
post #202
post #76

Vibe coded internet keeps getting better

You just paste the outage error codes back to the LLM and pray it's still working and can fix whatever went wrong!

When all the people forget to code for themselves, every LLM will code itself out of existence with that one last bug. One, after another.

Re: Tell HN: Azure outage

#242

Earlier quoted context omitted.

It's pretty unlikely. AWS published a public 'RCA' https://aws.amazon.com/message/101925/ . A race condition in a DNS 'record allocator' causing all DNS records for DDB to be wiped out. I'm simplifying a bit, but I don't think it's likely that Azure has a similar race condition wiping out DNS records on _one_ system than then propagates to all others. The similarity might just end at "it was DNS".

That RCA was fun. A distributed system with members that don't know about each other, don't bother with leader elections, and basically all stomp all over each other updating the records. It "worked fine" until one of the members had slightly increased latency and everything cascade-failed down from there. I'm sure there was missing (internal) context but it did not sound like a well-architected system at all.

Needs STONITH

Re: Tell HN: Azure outage

#243
post #231
post #224

Earlier quoted context omitted.

AFD is down quite often regionally in Europe for our services. In 50%+ the cases they just don‘t report it anywhere, even if its for 2h+.

Spam those Azure tickets. If you have a CSAM, build them a nice powerpoint telling the story of all your AFD issues (that's what they are there for). > In 50%+ the cases they just don‘t report it anywhere, even if its for 2h+. I assume you mean publicly. Are you getting the service health alerts?

I got a service health alert an hour after it started, saying the portal was having issues. Pretty useless and misleading.

Re: Tell HN: Azure outage

#244
post #197

Yesterday Amazon, today Microsoft. Are Google's cloud services going down tomorrow?

This is because Azure just copies everything AWS does. Google is a bit more innovative, they will have something else unexpected happen.

throwback to when they deleted a customer's entire account! https://arstechnica.com/gadgets/2024/05/google-cloud-acciden...

Re: Tell HN: Azure outage

#248
post #33

Update 16:57 UTC: Azure Portal Access Issues Starting at approximately 16:00 UTC, we began experiencing Azure Front Door issues resulting in a loss of availability of some services. In addition. customers may experience issues accessing the Azure Portal. Customers can attempt to use programmatic methods (PowerShell, CLI, etc.) to access/utilize resources if they are unable to access the portal directly. We have faile…

Yet another reason to move away from Front Door. We already had to do it for large files served from Blob Storage since they would cap out at 2MB/s when not in cache of the nearest PoP. If you’ve ever experienced slow Windows Store or Xbox downloads it’s probably the same problem. I had a support ticket open for months about this and in the end the agent said “this is to be expected and we don’t plan on doing anythin…

we are considering the same but because our website uses APEX domain we would need to move all DNS resolver to cloudfront right ? Does it have as a nice "rule set builder" as azure ?

Re: Tell HN: Azure outage

#249

Meanwhile the layoffs continue https://www.entrepreneur.com/business-news/microsoft-ceo-exp...

> [Satya Nadella] said that the company’s future opportunity was to bring AI to all eight billion people on the planet. But what if I don't want AI brought to me?

Real life Pluribus https://en.wikipedia.org/wiki/Pluribus_(TV_series)

Re: Tell HN: Azure outage

#250
All of my employers things are hosted on Azure and running just fine and didn't go down at all. Portal access has been fixed.

Doesn't seem to be too bad of an outage unless you were relying on Azure Front Door.

Post reply on HN