I'm a principal engineer at Microsoft. I barely program anymore • Hacker News Board

Re: I'm a principal engineer at Microsoft. I barely program anymore

#21

Post by Madmallard » Wed, Oct 22, 2025, 1:24 PM UTC

payneio wrote: post #13

proc0 wrote: post #3

I think we're going to see a negative impact on the software industry thanks to the LLM hype. There is a metric of LLMs which is hard to measure, and that is something like the quality of the solution, which includes how well the problem is abstracted, and how well the solution is decomposed in such a way that it becomes easily scalable, resilient, etc. The article shows how this is happening. The examples given are…

Yes. These are all the same points I used to believe until recently... in fact the article I write two months earlier was all about LLMs not being able to think like us. I still haven't squared how I can believe both things at the same time. The point of my article was to try to explain why I think otherwise now. Responding to your thoughts in sequence: - These systems can re-abstract and decompose things just fine.…

"- These systems can re-abstract and decompose things just fine. If you want to make it resilient or scalable it will follow whatever patterns you want to give it. These patterns are well known and are definitely in the training data for these models."

No? It absolutely does not do this correctly. It does what "looks" right. Not what IS right. And that ends up being wrong literally the majority of the time for anything even mildly complex.

" I'm sure we'll get some metrics soon on how functional these are for something like Windows, which, I believe is literally the world's single largest code base."

Now that's just not true at all. Windows doesn't even lay a finger to Google's code-base.

"and then make the results easy for a human to review."

This is in no way doable for anything not completely trivial from what an LLM produces. Software is genuinely hard and time-consuming if you want it to actually not be brittle and address the things it needs to and with trade-offs that are NOT detrimental to the future of your product.

Re: I'm a principal engineer at Microsoft. I barely program anymore

#22

Post by payneio » Wed, Oct 22, 2025, 1:38 PM UTC

Madmallard wrote: post #21

payneio wrote: post #13

Earlier quoted context omitted.

Yes. These are all the same points I used to believe until recently... in fact the article I write two months earlier was all about LLMs not being able to think like us. I still haven't squared how I can believe both things at the same time. The point of my article was to try to explain why I think otherwise now. Responding to your thoughts in sequence: - These systems can re-abstract and decompose things just fine.…

"- These systems can re-abstract and decompose things just fine. If you want to make it resilient or scalable it will follow whatever patterns you want to give it. These patterns are well known and are definitely in the training data for these models." No? It absolutely does not do this correctly. It does what "looks" right. Not what IS right. And that ends up being wrong literally the majority of the time for anythi…

How are you verifying your claims? I'm actually seeing results that you describe as being impossible.

Re: I'm a principal engineer at Microsoft. I barely program anymore

#23

Post by payneio » Wed, Oct 22, 2025, 1:48 PM UTC

proc0 wrote: post #19

payneio wrote: post #13

Earlier quoted context omitted.

Yes. These are all the same points I used to believe until recently... in fact the article I write two months earlier was all about LLMs not being able to think like us. I still haven't squared how I can believe both things at the same time. The point of my article was to try to explain why I think otherwise now. Responding to your thoughts in sequence: - These systems can re-abstract and decompose things just fine.…

Well, if it replaces all engineers, then I'm not up to date on the capabilities of the state of the art. So far I've just used the available commercial models. I quickly hit walls when I try to push its limits even a little. In theory, any prompt should result in a good output just as if I suggest it to an engineer. In practice I find that there are real limitations that require a lot of iterations and "handholding"…

So, what we do is automate the hand-holding. In your physics simulation example, you can have the system attempt to compile on every change and fix any errors it finds (we use strict linting, type-checking, compile errors, etc.); and you can provide a metric of "good" and have it check for that and revise/iterate as needed. What we've found particularly useful is breaking the problem into smaller pieces--"The Unix Philosophy" as the system is quite capable of extracting, composing, defining APIs, etc. over small pieces. Make larger things out of reliable smaller things like any reasonable architecture.

These things are not "creative"... they are just piecing together decent infrastructure and giving the "actor" the ability to use it.

Then break planning, design, implementation, testing, etc. apart and do the same for each phase--reduce "creativity" to process and the systems can follow the process quite nicely with minimal intervention.

Then, any time you do need to intervene, use the system to help you automate the next thing so you don't have to intervene in the same way again next time.

This is what we've been doing for months and it's working well.

Re: I'm a principal engineer at Microsoft. I barely program anymore

#24

Post by payneio » Wed, Oct 22, 2025, 1:50 PM UTC

Terretta wrote: post #18

payneio wrote: post #11

Earlier quoted context omitted.

What's wrong with "self promotion"? The point of this space has always been promoting projects. That's what Y Combinator is all about

What to Submit On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity. Please don't use HN primarily for promotion. It's ok to post your own stuff part of the time, but the primary use of the site should be for curiosity. https://news.ycombinator.com/news…

Thanks for the extract. I feel quite comfortable that my post is on-topic and gratifying. I understand others may disagree (and do in nearly every post on HN)

Re: I'm a principal engineer at Microsoft. I barely program anymore

#25

Post by payneio » Wed, Oct 22, 2025, 1:57 PM UTC

Madmallard wrote: post #20

If only AI was not completely and utterly useless for any unique problems for which there isn't extreme amounts of available training data. You know, something any competent programmer knows and has already known for years. And these problems end up being involved in basically every single non-trivial application and after not very long into development on those applications. If only AI didn't very readily and aggres…

I get it. I've been through cycles of this over the past three years, too. Used a lot of various tools, had a lot of disappointment, wasted a lot of time and money.

But this is the kinda the whole point of my post...

In our system, we added fact checking itself, comparing different approaches, summarizing and effectively utilizing the "wisdom of the crowd" (and it's success over time).

And it made it work massively better for even non-trivial applications.

Re: I'm a principal engineer at Microsoft. I barely program anymore

#26

Post by payneio » Wed, Oct 22, 2025, 2:00 PM UTC

Madmallard wrote: post #20

If only AI was not completely and utterly useless for any unique problems for which there isn't extreme amounts of available training data. You know, something any competent programmer knows and has already known for years. And these problems end up being involved in basically every single non-trivial application and after not very long into development on those applications. If only AI didn't very readily and aggres…

Also... "scammer and AI grifter"?? Damn dude. It's any early-stage open-source experiment result and, mostly, just talking about how it makes me question whether or not I'll be programming in the future. Nobody's asking for your money.

Re: I'm a principal engineer at Microsoft. I barely program anymore

#27

Post by Madmallard » Wed, Oct 22, 2025, 7:01 PM UTC

payneio wrote: post #26

Madmallard wrote: post #20

If only AI was not completely and utterly useless for any unique problems for which there isn't extreme amounts of available training data. You know, something any competent programmer knows and has already known for years. And these problems end up being involved in basically every single non-trivial application and after not very long into development on those applications. If only AI didn't very readily and aggres…

Also... "scammer and AI grifter"?? Damn dude. It's any early-stage open-source experiment result and, mostly, just talking about how it makes me question whether or not I'll be programming in the future. Nobody's asking for your money.

My last comment wasn't really directed at you it just reminded me of how I feel about the whole scene right now.

Re: I'm a principal engineer at Microsoft. I barely program anymore

#28

Post by Madmallard » Wed, Oct 22, 2025, 7:20 PM UTC

payneio wrote: post #25

Madmallard wrote: post #20

If only AI was not completely and utterly useless for any unique problems for which there isn't extreme amounts of available training data. You know, something any competent programmer knows and has already known for years. And these problems end up being involved in basically every single non-trivial application and after not very long into development on those applications. If only AI didn't very readily and aggres…

I get it. I've been through cycles of this over the past three years, too. Used a lot of various tools, had a lot of disappointment, wasted a lot of time and money. But this is the kinda the whole point of my post... In our system, we added fact checking itself, comparing different approaches, summarizing and effectively utilizing the "wisdom of the crowd" (and it's success over time). And it made it work massively b…

You're going to have to put quotes around "fact checking" if you're using LLMs to do it.

"comparing different approaches, summarizing and effectively utilizing the "wisdom of the crowd" (and it's success over time)"

I fail to see how this is defensible as well.

Re: I'm a principal engineer at Microsoft. I barely program anymore

#29

Post by proc0 » Thu, Oct 23, 2025, 3:30 AM UTC

payneio wrote: post #23

proc0 wrote: post #19

Earlier quoted context omitted.

Well, if it replaces all engineers, then I'm not up to date on the capabilities of the state of the art. So far I've just used the available commercial models. I quickly hit walls when I try to push its limits even a little. In theory, any prompt should result in a good output just as if I suggest it to an engineer. In practice I find that there are real limitations that require a lot of iterations and "handholding"…

So, what we do is automate the hand-holding. In your physics simulation example, you can have the system attempt to compile on every change and fix any errors it finds (we use strict linting, type-checking, compile errors, etc.); and you can provide a metric of "good" and have it check for that and revise/iterate as needed. What we've found particularly useful is breaking the problem into smaller pieces--"The Unix Ph…

Right, I can see how using an agentic system like that would go a long way. However there is a distinction between using AI models directly, and architecting a system in the context of this conversation, because it means the limitations of the models are being overcome by human engineers (and at scale, since this is hard outside of enterprise). If the models were intelligent enough this would not be needed.

So my claim of knowledge bases still stands. An agentic system designed by humans is still a system of knowledge bases that work with natural language, and of course their capability is impressive, but I remained unconvinced they can push the boundaries like a human can. That said, maybe pushing boundaries is not needed for the majority of applications out there, which I guess is fair enough and what we have now is good enough to make most human engineering obsolete. I guess we'll see in the near future.

Re: I'm a principal engineer at Microsoft. I barely program anymore

#30

Post by payneio » Thu, Oct 23, 2025, 8:36 PM UTC

lunias wrote: post #17

payneio wrote: post #12

Earlier quoted context omitted.

Yes. Please read it. I'm looking for collaborators. The links in this article point to recent work on Wild Cloud so you can see where it's currently at. Wild Cloud will is a network appliance that will let you set up a k8s cluster of Talos machines and deploy apps curated apps to it. It's meant to make self-hosting more accessible, which, yes, I think can help solve a lot of data sovereignty issues. I'm not sure what…

> I'm not sure what you mean by "barely programs" I felt like people were dumping on you in the comments for potentially "not coding a lot," but I checked and saw this is not true. > help solve a lot of data sovereignty issues I do agree with the need to solve data sovereignty issues, I'm not sure self-hosting is not already accessible; or that replicating the complication of cloud architecture makes it more accessib…

Ah! Gotcha. Thanks for the clarification.

The use cases I'm thinking that require cloud architecture are scaling up with GPUs (for self-hosted intelligence workloads). Also, Wild Cloud is meant to meet community needs more than individual needs (thought it will do that, too) so I'm imagining needing to scale horizontally more than just vertically. I would still recommend putting things like home-assistant or a home media server on SBCs.

It still is way more complex than I want it to be for a person to set up a local cluster, but I'm still hopeful I can make it simpler.