Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Decisions that eroded trust in Azure – by a former Azure Core engineer

isolveproblems.substack.com

Post reply on HN
Print view View original

151–160 of 697 posts

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#151

Post by refulgentis »

ryandrake wrote: post #143
ohyoutravel wrote: post #41

Earlier quoted context omitted.

Well, yeah, that’s what a board does, but I think the issue is whether it is customary to go to the board directly in this situation. The answer is a resounding NO. Very odd, but cool idea and approach.

Maybe naive, but why not? If it's a serious enough issue, and you're not getting anywhere through your management chain all the way up to the CEO, why is it novel to contact the people the CEO reports to? They're not royalty, they're other human beings who also eat, piss and fart like everyone else.

Before 6 years of Google I’d co-sign what you said, but it never ever plays out that way.

The law of the jungle is an iron law, make people around you feel bad, be a tattletale, and you’re choosing to be ostracized.

That said yr interlocutor disturbs me a bit because yes, they certainly will make it out to be a mental health episode. But the implicit deal there is “STFU. You can even take paid health leave.” It’s not healthy either. BigCo is insane I’ll never work for one again without outrageous comp.

You’d be stunned by even the simplest story. Ex. a year in some crazy shit was going down and my manager asked for my thoughts on a topic, I was honest and basically said “I don’t think it’s a good idea, but in my experience, raising issues involving people only raises more issues.” He swore up and down it wouldn’t be a problem, eventually made a deal I could email it to him privately. Next 1:1 with my area lead was horrible, them seeing red, hearing a mistranslated version of what I said, and I had 0 warning.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#152

Post by dh2022 »

nrds wrote: post #128
Anon1096 wrote: post #24

The post is so dramatized and clearly written by someone with a grudge such that it really detracts from any point that is trying to be made, if there is any. From another former Az eng now elsewhere still working on big systems, the post gets way way more boring when you realize that things like "Principle Group Manager" is just an M2 and Principal in general is L6 (maybe even L5) Google equivalent. Similarly Sev2 i…

I've worked at both Microsoft and Google in the past 6 years and the notion that msft "Principal" is equivalent to goog L5 is crazy.

Meaning Msft Principal is below L5? I got the same feedback from one of my friends who works at Google. She said quality of former MSFT engineers now working at Google was noticeably lower.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#153

Post by lokar »

dh2022 wrote: post #119
nope1000 wrote: post #5

> The direct corollary is that any successful compromise of the host can give an attacker access to the complete memory of every VM running on that node. Keeping the host secure is therefore critical. > In that context, hosting a web service that is directly reachable from any guest VM and running it on the secure host side created a significantly larger attack surface than I expected. That is quite scary

This is well documented: https://learn.microsoft.com/en-us/azure/virtual-machines/ins... Why would an Azure customer need to query this service at all? I was not aware this service even exists- because I never needed anything like it. AFAI can tell, this service tells services running on the VM what SKU the VM is. But how is this useful to the service? Any Azure users could tell how they use IMDS? Thanks!

To have a new vm configure itself at boot

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#154

Post by lmm »

mpyne wrote: post #75
abtinf wrote: post #50

Earlier quoted context omitted.

> risks to national security …really? Really. Apparently the Secretary of War agrees with him.

In fairness the SECWAR is hardly a computing expert. But in this case the SECWAR has been properly advised. If anything it's astonishing that a program whereby China-based Microsoft engineers telling U.S.-based Microsoft engineers specific commands to type in ever made it off the proposal page inside Microsoft, accelerated time-to-market or not. It defeats the entire purpose of many of the NIST security controls that…

Being compliant with the letter of the requirements at 1/3 of the cost is absolutely an idea that sells itself.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#155

Post by jimbobimbo »

dh2022 wrote: post #119
nope1000 wrote: post #5

> The direct corollary is that any successful compromise of the host can give an attacker access to the complete memory of every VM running on that node. Keeping the host secure is therefore critical. > In that context, hosting a web service that is directly reachable from any guest VM and running it on the secure host side created a significantly larger attack surface than I expected. That is quite scary

This is well documented: https://learn.microsoft.com/en-us/azure/virtual-machines/ins... Why would an Azure customer need to query this service at all? I was not aware this service even exists- because I never needed anything like it. AFAI can tell, this service tells services running on the VM what SKU the VM is. But how is this useful to the service? Any Azure users could tell how they use IMDS? Thanks!

Managed identity is enabled via that endpoint, for example.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#156

Post by deathanatos »

dh2022 wrote: post #119
nope1000 wrote: post #5

> The direct corollary is that any successful compromise of the host can give an attacker access to the complete memory of every VM running on that node. Keeping the host secure is therefore critical. > In that context, hosting a web service that is directly reachable from any guest VM and running it on the secure host side created a significantly larger attack surface than I expected. That is quite scary

This is well documented: https://learn.microsoft.com/en-us/azure/virtual-machines/ins... Why would an Azure customer need to query this service at all? I was not aware this service even exists- because I never needed anything like it. AFAI can tell, this service tells services running on the VM what SKU the VM is. But how is this useful to the service? Any Azure users could tell how they use IMDS? Thanks!

> Why would an Azure customer need to query this service at all? I was not aware this service even exists- because I never needed anything like it.

The "metadata service" is hardly unique to Azure (both GCP & AWS have an equivalent), and it is what you would query to get API credentials to Azure (/GCP/AWS) service APIs. You can assign a service account² to the VM¹, and the code running there can just auto-obtain short-lived credentials, without you ever having to manage any sort of key material (i.e., there is no bearer token / secret access key / RSA key / etc. that you manage).

I.e., easy, automatic access to whatever other Azure services the workload running on that VM requires.

¹and in the case of GCP, even to a Pod in GKE, and the metadata service is aware of that; for all I know AKS/EKS support this too

²I am using this term generically; each cloud provider calls service accounts something different.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#157

Post by lokar »

dh2022 wrote: post #123
grensley wrote: post #107

Earlier quoted context omitted.

Absolutely textbook "Brilliant Jerk". Dude just whines and whines and whines. If you're so good, why can't you get anybody to work with you?

I did not get that impression at all. He mentioned quite a few conversations with partner level employees, technical fellow, principal managers. The impression I got is he tried to fix things, but the mess is so widespread and decision makers are so comfortable in this mess that nobody wants to stick their necks out and fix things. I got strong NASA Challenger vibes when reading this story…

My read is he was not Sr enough in the org to drive any effort to improve things, and could not get someone who was to do it either.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#158

Post by doubled112 »

macNchz wrote: post #129
yoyohello13 wrote: post #27

I don't know if any of this is true, but as a user of Azure every day this would explain so much. The Azure UI feels like a janky mess, barely being held together. The documentation is obviously entirely written by AI and is constantly out of date or wrong. They offer such a huge volume of services it's nearly impossible to figure out what service you actually want/need without consultants, and when you finally get t…

I’ve created a bunch of fresh Azure accounts over the past few years and each time I’ve found myself sitting there dumbfounded anew at how garbage the experience is. There has been weird broken jank at just about every step of the process at one point or another. Like, I’m a serious person trying to set something up for a production workload, and multiple times along the way to just having a working account that I ca…

I've joked that on some services, when you're clicking buttons, you're actually opening tickets that a human needs to action.

That scenario is an example. You complete an action on a web page and nothing works. You make no further changes and hours later it works perfectly. Your human wasn't fast enough that day.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#159

Post by axelriet »

VladVladikoff wrote: post #132
vintagedave wrote: post #9

What are we reading here? These are extraordinary statements. Also with apparent credibility. They sound reasonable. Is this a whistleblower or an ex employee with a grudge? The appearance is the first. Is it? They’ve put their name to some clear and worrying statements. > On January 7, 2025… I sent a more concise executive summary to the CEO. … When those communications produced no acknowledgment, I took the customa…

I am sort of confused how NDA and such agreements employees sign would allow for an employee to post such an article without being sued by Microsoft?

Interesting point. Time will tell.

Re: Decisions that eroded trust in Azure – by a former Azure Core engineer

#160

Post by jmogly »

dh2022 wrote: post #119
nope1000 wrote: post #5

> The direct corollary is that any successful compromise of the host can give an attacker access to the complete memory of every VM running on that node. Keeping the host secure is therefore critical. > In that context, hosting a web service that is directly reachable from any guest VM and running it on the secure host side created a significantly larger attack surface than I expected. That is quite scary

This is well documented: https://learn.microsoft.com/en-us/azure/virtual-machines/ins... Why would an Azure customer need to query this service at all? I was not aware this service even exists- because I never needed anything like it. AFAI can tell, this service tells services running on the VM what SKU the VM is. But how is this useful to the service? Any Azure users could tell how they use IMDS? Thanks!

Mainly for getting managed-identity access tokens for Azure APIs. In AWS you can call it to get temporary credentials for the EC2’s attached IAM role. In both cases - you use IMDS to get tokens/creds for identity/access management.

Client libraries usually abstract away the need to call IMDS directly by calling it for you.

Post reply on HN