Live data from Hacker News

Decisions that eroded trust in Azure – by a former Azure Core engineer

isolveproblems.substack.com

491–500 of 697 posts

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#491

Earlier quoted context omitted.

This isn't incentivized in corporate environment. Noticed how "the talent left after the launch" is mentioned in the article? Same problem. You don't get rewarded for cleaning up mess (despite lip service from management) nor for maintaining the product after the launch. Only big launches matter. The other corporate problem is that it takes time before the cleanup produces measurable benefits and you may as well get…

This is the root of the issue. For something like Azure, people are nor fungible. You need to retain them for decades, and carefully grow the team, training new members over a long period until they can take on serious responsibilities. But employees are rewarded for showing quick wins and changing jobs rapidly, and employers are rewarded for getting rid of high earners (i.e. senior, long-term employees).

[dead]

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#492
post #46

What are we reading here? These are extraordinary statements. Also with apparent credibility. They sound reasonable. Is this a whistleblower or an ex employee with a grudge? The appearance is the first. Is it? They’ve put their name to some clear and worrying statements. > On January 7, 2025… I sent a more concise executive summary to the CEO. … When those communications produced no acknowledgment, I took the customa…

Yeah I thought that was extreme. An engineer going to the board of any corporation let alone Microsoft is not normal or customary IME. That could explain why they got no response.

It's a baffling flaw in human nature. The board should have cared about these issues, but in practice communications to and from the board are tightly controlled, and communications outside of those constraints are discarded.

This occurs whether or not it makes sense. Machiavelli actually warns about the specifically: if someone else controls access to you and communication with you, they have real leverage over you.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#493
post #242

Earlier quoted context omitted.

> I submitted several bug fixes and refactoring, notably using smart pointers, but they were rejected for fear of breaking something. And that, my friends, is why you want a memory safe language with as many static guarantees as possible checked automatically by the compiler.

Language choices won't save you here. The problem is organizational paralysis. Someone sees that the platform is unstable. They demand something be done to improve stability. The next management layer above them demands they reduce the number of changes made to improve stability.

Usually this results in approvals to approve the approval to approve making the change. Everyone signed off on a tower of tax forms about the change, no way it can fail now! It failed? We need another layer of approvals before changes can be made!

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#494
This is pretty damning, if half of it is true. I don’t work at Microsoft and I don’t have the knowledge to judge the reliability of Azure, but I do have friends who work as users of Azure and the words are not kind, especially the new Fabric database which is said to be crazy to pick for production at this stage — while MSFT switched the certification to Fabric already, pushing its customers to use it.

I’ll never work in a company that uses Azure as its main cloud services, just for the sake of quiet nights.

I do wonder what does it look like inside AWS and GCP, though. Is it the same level of chaos, but just because they started early they got more success? If that’s the case, maybe we can conclude, that very large cloud operation is not sustainable under the current company structure — because either the technical knowledge required is too dense, but companies won’t be able to retain workers, or because companies are forced to join the horn of the marketeers, eventually.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#495
My comment history here is full of complaints about MS Teams, the chat app. It suffers from the "re-use every existing MS tech" problem. Building it on top of SharePoint I'm pretty sure resulted in it's top problems over the years (some fixed): - search sucked - can't scroll back to old messages - couldn't do private channels - the very concept of 1:1 teams to SharePoint site, resulting in a millions teams when all you really wanted was a channel - can't rename teams or channels - couldn't do private channels

I'm sure many more I didn't catch. These are all observations from outside, I've never worked at MS

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#496

I think this is especially problematic (from Part 4 at https://isolveproblems.substack.com/p/how-microsoft-vaporize... ): "The team had reached a point where it was too risky to make any code refactoring or engineering improvements. I submitted several bug fixes and refactoring, notably using smart pointers, but they were rejected for fear of breaking something." Once you reach this stage, the only escape is to first…

if the service is so shitty, why are people paying so much fucking money for it? is microsoft committing an accounting fraud?

I have worked at two retail companies where AWS was a no no. They didn't want to have anything depending on a competitor(Amazon). So they went the Azure route.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#497

Earlier quoted context omitted.

This isn't incentivized in corporate environment. Noticed how "the talent left after the launch" is mentioned in the article? Same problem. You don't get rewarded for cleaning up mess (despite lip service from management) nor for maintaining the product after the launch. Only big launches matter. The other corporate problem is that it takes time before the cleanup produces measurable benefits and you may as well get…

This is the root of the issue. For something like Azure, people are nor fungible. You need to retain them for decades, and carefully grow the team, training new members over a long period until they can take on serious responsibilities. But employees are rewarded for showing quick wins and changing jobs rapidly, and employers are rewarded for getting rid of high earners (i.e. senior, long-term employees).

This is a human problem. We humans praise the doctors that can put the patients with terminal illnesses alive for extended periods, but ignore those who tell us the principles to prevent getting those illnesses in the first place. We throw flowers and money to doctors who treat cancers, but do we do the same to the ones who tell us principles to avoid cancers? No.

The same for MSFT or any other similar problem. Humans only care when the house is on fire — in the modern Capitalism it means the stock goes down 50%, and then they will have the will to make changes.

That’s also why reforms rarely succeeded, and the ones that succeeded usually follows a huge shitstorm when people begged for changes.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#498

Earlier quoted context omitted.

> For something like Azure, people are nor fungible What I've learned from a decade in the industry is that talent is never fungible in low-demand areas. It's surprisingly hard to find people that "get it" and produce something worthwhile together.

I would say "systems design" rather than low-demand. People who can "reduce" a big system to build on a few simple concepts are few and far between. Most people just add more stuff instead.

When things become too complicated, no one dares to make new systems. And if you don’t make new systems ofc you have to learn system design the other way around — by fixing every bug of existing systems.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#499

Earlier quoted context omitted.

I was waiting for that comment :) Remember that everybody, eventually, calls into code written in C.

It’s worse than that. Eventually everybody calls into code that hits hardware. That is the level that the compiler (ironically?) can no longer make guarantees. Registers change outside the scope of the currently running program all the time. Reading a register can cause other registers on a chip to change. Random chips with access to a shared memory bus can modify the memory that the comipler deduced was static. Ther…

Sure, but eliminating bugs isn't a binary where you either eliminate all of them or it's a useless endeavor. There's a lot of value in eliminating a lot of bugs, even if it's not all of them, and I'd argue that empirically Rust does actually make it easier to avoid quite a large number of bugs that are often found in C code in spite of what you're saying.

To be clear, I'm not saying that I think it would necessarily be a good idea to try to rewrite an existing codebase that a team apparently doesn't trust they actually understand. There are a lot of other factors that would go into deciding to do a rewrite than just "would the new language be a better choice in a vaccuum", and I tend to be somewhat skeptical that rewriting something that's already widely being used will be possible in a way that doesn't end up risking breaking something for existing users. That's pretty different from "the language literally doesn't matter because you can't verify every possible bug on arbitrary hardware" though.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#500
Is it just me or does this describe most of Microsoft software at the moment? I tried to sign into my personal microsoft account to setup an oauth flow and I was greated by an infinity repeating error dialog about some internal service that had failed.

At work, I use outlook. The number of times I've gotten caught in an auth loop where I enter again and again my creds + tfa only for the screen to flicker and start all over again.

Complete garbage.

Post reply on HN