Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Engineering management after the cost of code collapsed

karimjedda.com

Post reply on HN
Print view View original

121–130 of 217 posts

Re: Engineering management after the cost of code collapsed

#121

Post by chickensong »

> Sorting requires a model of how your org actually behaves: trust relationships, hallway knowledge, the consequences of past decisions. Almost none of this is written down.

This is a long-standing problem related to operational excellence and politics. I expect this will improve with AI adoption and integration. You can't get an exec to create a decision record and commit it to git. Managers have incentive to sequester information.

Engineering already has the discipline (maybe) and abilities to solve the problem. Version control, change control, ADRs, logging, structured docs, etc... We can trace an inbound packet or call through the entire stack. Management can't/won't do anything remotely close. 1-to-1 emails, meeting minutes, stale Word docs is the standard for most.

Inserting LLMs as the interface, and/or plugging into existing interfaces like email, is going to change things. Finally it will be possible to capture more institutional knowledge, without trying to teach an old dog new tricks.

Re: Engineering management after the cost of code collapsed

#122

Post by consumer451 »

drTobiasFunke wrote: post #108
dbingham wrote: post #36

I think most of this is correct, in spite of potentially being built on a bad assumption. The assumption is that LLMs should be writing the code and human engineers reviewing and verifying the LLM output. And that this pushes the cost of producing down. And I fundamentally disagree with that. Every time I ask LLMs to write code, even with Opus 4.8 (haven't tried it with Opus 5 yet), what I get ends up being totally r…

“Not in speed, but in quality” — unfortunately though, not a single CEO, executive, company, VC, investor or anyone with the power to make decisions cares about quality instead of speed.

I agree. Also, what does "code quality" even mean in the age of agentic dev? If the code works, and is secure, what else matters?

I do know what good code looks like, but does that even matter anymore? All I know is that now, I get to focus on endless UX polish, which is the only thing the matters.

I feel like we are living through something like the Protestant Reformation, where priests once spoke Latin, and then started to speak in plain local language. The old guard did not like this.

Re: Engineering management after the cost of code collapsed

#123

Post by [deleted] »

jtorsella wrote: post #114
CharlesW wrote: post #71

Earlier quoted context omitted.

Pangram's marketing always reminds me of Anchorman's Sex Panther cologne: "They've done studies, you know. Sixty percent of the time, it works every time." Pangram's "100% AI generated" claims are right 65% of the time. https://link.springer.com/article/10.1007/s40979-026-00226-w

Did you read the paper you are linking to? It records a 0% false positive rate for evaluation of human-authored controls, and fewer than 5% of the hybrid and humanized papers had their AI levels overestimated by pangram. It is not completely clear from the data, but it seems that depending on whether the n=2 overestimates were “100% ai generated” assessments, the study you link says pangram’s 100% AI assessments were…

[deleted]

Re: Engineering management after the cost of code collapsed

#124

Post by lordnacho »

twister2920 wrote: post #99
lordnacho wrote: post #94

Earlier quoted context omitted.

> The other thing that the "LLMs write code camp" misunderstands is that writing was never the bottleneck. Understanding was. And understanding the code is still the bottleneck. But understanding is truly gained during the writing loop. The understanding you gain from pure reading or code review is marginal compared to the understanding you gain while writing. This was my stance a couple of years ago, but now I've gi…

> And each time you do one of those, there's a decent chance you did something even more trivial like forgetting a semicolon or calling the wrong function. how did you decide to pick the most trivial kind regression for this example? do you compile your code before checking it in? > A human junior programmer could perfectly well understand what this meant, but he would have to go through all of the above to get to th…

> how did you decide to pick the most trivial kind regression for this example?

Why would this be a regression? You might just be writing a new line of code.

> do you compile your code before checking it in?

Well obviously. That is generally how you discover that a semicolon is missing.

> the main efficiency you have described here is offloading the verification of a change onto the LLM. that is the bottleneck. readers can decide whether a non-deterministic statistical model is a good tool for this job

No, it's the time between you deciding something needs to be done, and it being done, that is the bottleneck. You cannot avoid trying to compile the code and testing it. Now you can get to that test without paying attention, which is time you can use productively.

> readers can decide whether a non-deterministic statistical model is a good tool for this job

Somehow, the non-deterministic model has built me the deterministic code that I want, very fast, pretty much all the time. A year ago it would get stuck. Now it doesn't, for me at least, and for competent programmers that I know.

> the best programmers understand that their job is to automate workflows, and that includes their own. if you're worried about missing a semicolon, I'm sorry to say that's a skill issue

Well yeah, and I've automated my workflows completely. I don't have the problems I used to have. If you haven't caught on to the new way of working, well, that's a skill issue...

Re: Engineering management after the cost of code collapsed

#125

Post by YZF »

dbingham wrote: post #36

I think most of this is correct, in spite of potentially being built on a bad assumption. The assumption is that LLMs should be writing the code and human engineers reviewing and verifying the LLM output. And that this pushes the cost of producing down. And I fundamentally disagree with that. Every time I ask LLMs to write code, even with Opus 4.8 (haven't tried it with Opus 5 yet), what I get ends up being totally r…

In my domain since circa Opus 4.5 LLMs write code as good as most engineers given well defined small enough chunk of work. They refactor. They write tests. These days LLM also debug/troubleshoot better than most engineers.

Are they as good as handcrafted code by 0.1% of top software engineers. Generally no. But neither is 99.9% of real code.

LLMs also are good at code reviews. What they'll miss is often the big picture but they can still catch plenty of issues. I still want to see a human in the loop in my domain.

Totally agree that writing the code was never the bottleneck. We're not seeing massive productivity gains even if some code is written faster. It's not just about understanding but also various other activities that happen in large companies and teams.

Also agree LLMs can be used to gain quality but realistically most orgs are going to aim for "fixed or decreasing" quality at lower costs.

Re: Engineering management after the cost of code collapsed

#127

Post by YZF »

SoftTalker wrote: post #68
dbingham wrote: post #36

I think most of this is correct, in spite of potentially being built on a bad assumption. The assumption is that LLMs should be writing the code and human engineers reviewing and verifying the LLM output. And that this pushes the cost of producing down. And I fundamentally disagree with that. Every time I ask LLMs to write code, even with Opus 4.8 (haven't tried it with Opus 5 yet), what I get ends up being totally r…

How do you handle the fact that an LLM can't seem to help itself from disgorging page after page of words no matter what it's asked to do? I've never seen an LLM say "this looks good as-is; I would not spend any more time on it; what's next?" it will always seem to suggest using another pattern or additional abstractions or other yak shaving. But to be fair human code reviews have the same problem. It's like reviewer…

I've totally had latest models just say "here are minor nits but this is good to ship" when reviewing code. Claude is a bit more verbose usually but Codex by default is pretty terse. My experience anyways. They also take pushback on suggestions (this isn't actually a problem because X) and they'll agree and say ship it (hopefully only when you're right and it's not a problem ;) ).

Re: Engineering management after the cost of code collapsed

#128

Post by theptip »

gste wrote: post #107
dbingham wrote: post #36

I think most of this is correct, in spite of potentially being built on a bad assumption. The assumption is that LLMs should be writing the code and human engineers reviewing and verifying the LLM output. And that this pushes the cost of producing down. And I fundamentally disagree with that. Every time I ask LLMs to write code, even with Opus 4.8 (haven't tried it with Opus 5 yet), what I get ends up being totally r…

I think this is outdated. If you follow spec-driven development, get the model to do all the planning work upfront, review and iterate the plan, write clear markdown file documentation on the abstractions and patterns you want to follow, then you have every opportunity to tell the model how you want it to write the code. If you use Opus or Fable 5 it will then write the code better and faster than you will.

Agreed, all of the staff/principal engineers I work with are ~99%+ AI generated code, and increasing business value delivered as a result. This is on planet scale infra not CRUD apps. (And yes, you do need to carefully review the output and give steers/corrections. It’s still faster.)

At this point if you can’t get the agent to write good code then either I) you are in a very specific niche (like Karpathy trying to write NanoGPT) that is extremely out-of-distribution, or II) skill issue, you need to learn how to prompt better.

It’s fine to have a skill gap! Just don’t delude yourself that the tools are bad and everyone claiming they are good is wrong.

Re: Engineering management after the cost of code collapsed

#129

Post by CharlesW »

jtorsella wrote: post #114
CharlesW wrote: post #71

Earlier quoted context omitted.

Pangram's marketing always reminds me of Anchorman's Sex Panther cologne: "They've done studies, you know. Sixty percent of the time, it works every time." Pangram's "100% AI generated" claims are right 65% of the time. https://link.springer.com/article/10.1007/s40979-026-00226-w

Did you read the paper you are linking to? It records a 0% false positive rate for evaluation of human-authored controls, and fewer than 5% of the hybrid and humanized papers had their AI levels overestimated by pangram. It is not completely clear from the data, but it seems that depending on whether the n=2 overestimates were “100% ai generated” assessments, the study you link says pangram’s 100% AI assessments were…

My citation was correct. To question to ask yourself: For my use case, is it okay that Pangram can't reliably tell the difference between "100% AI generated" and "AI assisted"?

Re: Engineering management after the cost of code collapsed

#130

Post by piker »

theptip wrote: post #128
gste wrote: post #107

Earlier quoted context omitted.

I think this is outdated. If you follow spec-driven development, get the model to do all the planning work upfront, review and iterate the plan, write clear markdown file documentation on the abstractions and patterns you want to follow, then you have every opportunity to tell the model how you want it to write the code. If you use Opus or Fable 5 it will then write the code better and faster than you will.

Agreed, all of the staff/principal engineers I work with are ~99%+ AI generated code, and increasing business value delivered as a result. This is on planet scale infra not CRUD apps. (And yes, you do need to carefully review the output and give steers/corrections. It’s still faster.) At this point if you can’t get the agent to write good code then either I) you are in a very specific niche (like Karpathy trying to w…

How is a GPT implementation “extremely out of distribution” but “planet scale infra” isn’t? That’s got me totally confused about your point that I was taking seriously.
Post reply on HN