MS Azure down: An emerging issue is being investigated • Hacker News Board

Re: MS Azure down: An emerging issue is being investigated

#121

Post by aynsof » Fri, Apr 02, 2021, 6:47 AM UTC

grey-area wrote: post #118

aynsof wrote: post #48

Earlier quoted context omitted.

Do you think locally managed systems are immune to outages? Or that governments are capable of resourcing their teams sufficiently to do a better job at availability than Microsoft, Google, or Amazon?

On the evidence, yes. Cloud providers frequently have outages lasting multiple hours affecting millions of clients. Some governments manage to run critical servers without downtime.

While I accept of course that there are government-owned systems that have uptime measured in years (decades?), I think that extrapolating that into an argument about the overall reliability of those systems falls into survivorship bias.

But if I had to make an a priori prediction about which systems would have the better reliability over the long term - cloud-based or those run out of a corporate DC - my money would be on the cloud-based systems.

The cloud providers are going to have better management of power/network/hardware than the most mature government agency, simply because it's a core capability for them.

Re: MS Azure down: An emerging issue is being investigated

#122

Post by andrewnicolalde » Fri, Apr 02, 2021, 8:01 AM UTC

austinheap wrote: post #71

cm2187 wrote: post #61

Earlier quoted context omitted.

I can think of a few good reasons. Strategic (you don’t want to hand over to a foreign power your critical infrastructures). Diversification (if all your banks run on aws, the day aws goes down you don’t have a banking system anymore). Not being at the mercy of a capricious tech billionaire (what happened to Parler could very well happen to a state if the said billionaire doesn’t like your policy).

I’m unaware of any Core Banking Systems (CBS) that runs on a cloud provider with the exception being Finastra (Azure). Other parts of retail banking stacks? Sure. Not their cores.

Thought Machine’s Vault targets the big 3 IIRC.

Re: MS Azure down: An emerging issue is being investigated

#123

Post by FAKEDETECTOR » Fri, Apr 02, 2021, 8:18 AM UTC

People are still using that?

Think about it - what kind of work does involve "reliable infra" today?

Instead of knowing one Linux system well and maintaining multiple servers in different data centers with different providers you now have the stupid overhead of multiple cloud infra providers with all their lockin-pitfalls and incompatible specialities.

The promises of cloud have not been delivered.

It is all fake.

Re: MS Azure down: An emerging issue is being investigated

#124

Post by Jouvence » Fri, Apr 02, 2021, 10:07 AM UTC

qbasic_forever wrote: post #39

Scene_Cast2 wrote: post #32

I found out about this because I almost lost a password with bitwarden just now. Their "add new password" prompt is failing silently.

Yikes! That's not a good failure mode at all.

The same thing happens (or at least, did happen last year) if you don't have the correct app permissions set when using Bitwarden on a mobile device.

Re: MS Azure down: An emerging issue is being investigated

#125

Post by grey-area » Fri, Apr 02, 2021, 10:19 AM UTC

aynsof wrote: post #121

grey-area wrote: post #118

Earlier quoted context omitted.

On the evidence, yes. Cloud providers frequently have outages lasting multiple hours affecting millions of clients. Some governments manage to run critical servers without downtime.

While I accept of course that there are government-owned systems that have uptime measured in years (decades?), I think that extrapolating that into an argument about the overall reliability of those systems falls into survivorship bias. But if I had to make an a priori prediction about which systems would have the better reliability over the long term - cloud-based or those run out of a corporate DC - my money would…

Change is a leading cause of outages. The cloud providers are constantly changing and upgrading their services, and constantly have downtime ranging from minutes to hours.

In contrast UK gov servers like hmrc.gov.uk (for example) are pretty stable, I can't remember the last time I heard about an outage.

Cloud providers are certainly better at some things (like staying up to date with latest tech), but I'd contend reliability is not one of those things based on their prominent and frequent outages (at least several a year).

Re: MS Azure down: An emerging issue is being investigated

#126

Post by tkzed49 » Fri, Apr 02, 2021, 10:46 AM UTC

FAKEDETECTOR wrote: post #123

People are still using that? Think about it - what kind of work does involve "reliable infra" today? Instead of knowing one Linux system well and maintaining multiple servers in different data centers with different providers you now have the stupid overhead of multiple cloud infra providers with all their lockin-pitfalls and incompatible specialities. The promises of cloud have not been delivered. It is all fake.

a lot of pretty big business (including the cloud providers themselves) run very successful services on cloud servers. I don't see how it's fake.

Re: MS Azure down: An emerging issue is being investigated

#127

Post by faeriechangling » Fri, Apr 02, 2021, 11:56 AM UTC

thetinguy wrote: post #76

austinheap wrote: post #71

Earlier quoted context omitted.

I’m unaware of any Core Banking Systems (CBS) that runs on a cloud provider with the exception being Finastra (Azure). Other parts of retail banking stacks? Sure. Not their cores.

The major cloud providers all aim for 99.999% uptime. Keep in mind that 99.99% uptime means ~4 minutes of downtime a month. I think there are other reasons that banks may not want or have the ability to run their core services on the cloud.

If that's what they're aiming for they're sure missing their targets an awful lot. I guess that I'm proud that they're aiming for the stars!

Re: MS Azure down: An emerging issue is being investigated

#128

Post by abarringer » Fri, Apr 02, 2021, 1:40 PM UTC

spydum wrote: post #51

fartcannon wrote: post #42

Earlier quoted context omitted.

And here it is, the main problem with outsourcing critical infrastructure. Remote server providers like Microsoft Azure should at most be used as twins/redundant systems to a locally managed system. Governments of the world: pay your IT people more money to prevent brain drain.

So you think local IT can acheive the same high availability and elasticity? Sorry that isn't usually my experience. Lots of anecdotal local IT get lucky, but on average I think this is the wrong lesson to learn.

High availability? Certainly. Elasticity? Not as well.

The right lesson is "Be a person that strives for excellence in all spheres of life".

I've been doing "Local IT" since '96 and I spun up an EC2 instance the day I saw it announced on slashdot and have been using both ever since. Both excel in some ways in the hands of good people.

Anyone thinking "I'll move to the cloud (or to on-prem) and all my problems will magically go away" is fooling themselves. If you suck at on-prem those underlying issues will carry into the cloud. If you have excellence in a well run on-prem install you'll experience great benefits leveraging the cloud.

For instance, Last month I was discussing a "move to the cloud" with a bank CTO. They had an unreliable on-prem network and moved to the cloud.. and they just discovered after suffering an outage in the cloud what an "availability zone" was. The same attitudes that made their on-prem unreliable, insecure, expensive will make the cloud the same way for them.

Re: MS Azure down: An emerging issue is being investigated

#129

Post by DougN7 » Fri, Apr 02, 2021, 1:54 PM UTC

aynsof wrote: post #48

fartcannon wrote: post #42

Earlier quoted context omitted.

And here it is, the main problem with outsourcing critical infrastructure. Remote server providers like Microsoft Azure should at most be used as twins/redundant systems to a locally managed system. Governments of the world: pay your IT people more money to prevent brain drain.

Do you think locally managed systems are immune to outages? Or that governments are capable of resourcing their teams sufficiently to do a better job at availability than Microsoft, Google, or Amazon?

I’m starting to think the answer is yes. My small systems have not had outages like Azure has.

Re: MS Azure down: An emerging issue is being investigated

#130

Post by nonfamous » Fri, Apr 02, 2021, 2:57 PM UTC

orf wrote: post #55

> Microsoft rerouted traffic to our resilient DNS capabilities and are seeing improvement in service availability At least use well formed English when posting your update...

It may be British English, where companies are collective nouns and pluralised.