Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

arxiv.org

Post reply on HN
Print view View original

171–180 of 230 posts

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#171

Post by abalashov »

ryan_lane wrote: post #119
archagon wrote: post #118

Earlier quoted context omitted.

That’s why we abstract the useful code away as libraries, frameworks, etc. AI is not an abstraction.

You generally need to wire libraries in to your service, and you may be using the library in a slightly different way than normal. AIs are perfectly capable of doing this. Back to the original point, though: most software engineering work isn't novel. Most people are working on slightly different iterations of the same thing, but with the aim of different products. You can have completely different products that use…

It depends on the programming. However, you're right: another Angular reinsurance accounting application doesn't pose much of a challenge for LLMs.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#172

Post by __mharrison__ »

dominotw wrote: post #110
guhcampos wrote: post #93

I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet. In my experience, for anything complex enough, I have to start adding more and more constraints,…

[flagged]

99% of coders don't need to generate anything novel.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#173

Post by fredcallagan »

Very interesting paper and I must say that I totally agree with it. But also that is something that is not new. I would say that the initial expectation is a bit off. I never expected that picking up any agentic coding solution, drop it in a project and fire at it a list of tasks would just magically work and follow a project pre-defined constraints. I do not believe that any agentic coding stack comes out of the box capable of this. Agents still need proper mechanics to understand the context, constraints and objectives reliably and that's still a work in progress as we can see by the constant updates on tools and skills and processes from Leading AI labs. They are now trying to fill that additional layer, which by the way could be much more profitable then bare model and token consumption. I would also argue that current OS models, like the ones tested, if properly driven can already produce production code following the desired constraints.

What has been you experience? What has your production code looked like in recent months?

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#174

Post by __mharrison__ »

yomismoaqui wrote: post #6

Also they used languages with dynamic typing like Python & JS. In my experience a statically typed codebase is easier to maintain for humans so maybe it is also for agents. When using Codex/Claude Code with Go code I cannot count the times the agent does some change, runs a build to check for errors, find some and fix them.

Just have your harness rules add types and run ty after every change. Models handle Python typing quite well these days.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#175

Post by pron »

The situation is worse. Not only do agents have more difficulty under "structural constraints", but structural constraints may need to change, and agents are even worse at that.

When designing a system or a component we have ideas that form invariants. Sometimes the invariant is big, like a certain grand architecture, and sometimes it’s small, like the selection of a data structure. Except, eventually, you’ll want to add a feature that clashes with that invariant. At that point there are usually three choices:

- Don’t add the feature. The invariant is a useful simplifying principle and it’s more important than the feature; it will pay dividends in other ways.

- Add the feature inelegantly or inefficiently on top of the invariant. Hey, not every feature has to be elegant or efficient.

- Go back and change the invariant. You’ve just learnt something new that you hadn’t considered and puts things in a new light, and it turns out there’s a better approach.

Often, only one of these is right. Often, at least one of these is very, very wrong, and with bad consequences. Even when they are able to follow constraints, agents are terrible at identifying when the constraints need to change.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#176

Post by coldtea »

sirsinsalot wrote: post #168
coldtea wrote: post #154

Earlier quoted context omitted.

I don't think you understand how LLMs work. They're not merely re-arranging pre-existing blocks of code. And they have been shown to develop emergent properties that weren't in their training set time and again. They generate novel things as much as the average programmer (which works after himself having practice, exposure to codebases, and training, and reading API documentation, and such) generates novel things.

Not to be pedantic but the emergent properties are in the training set, and thus the model and algorithm. There's no magic coming from the universe. What makes the behavior emergent is that it can't be predicted at training time. The emergent and unpredictable output is the result of massive vector complexity being encoded.

>Not to be pedantic but the emergent properties are in the training set, and thus the model and algorithm. There's no magic coming from the universe.

You are either being pedantic or missing the point of emergent however.

Yes, it's not some novel unforeseen thing, like a magical Marvel Universe material or some unknown to humanity mode of thinking. Same way when people make something new they still recombine known words, or colors, or physical things in the universe.

It is however new capabilities that is not explicitly in the training set and can't be predicted by it. Like teaching something only calculus training materials and it figures out boolean algebra.

>The emergent and unpredictable output is the result of massive vector complexity being encoded

As opposed to what in humans? God given revelation?

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#177

Post by abalashov »

pron wrote: post #175

The situation is worse. Not only do agents have more difficulty under "structural constraints", but structural constraints may need to change, and agents are even worse at that. When designing a system or a component we have ideas that form invariants. Sometimes the invariant is big, like a certain grand architecture, and sometimes it’s small, like the selection of a data structure. Except, eventually, you’ll want to…

Despite my very limited enthusiasm for agentic coding, I have some experience with it, and my experience matches what you say perfectly. This is one of the seams that runs between pattern recognition and reasoning, and--despite the marketing claims around chain of thought--LLMs do not reason, not in the slightest.

All attempts to make them appear to reason are basically recursive confinement efforts by the harness, to try to get the lightning into the bottle.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#178

Post by dominotw »

__mharrison__ wrote: post #172
dominotw wrote: post #110

Earlier quoted context omitted.

[flagged]

99% of coders don't need to generate anything novel.

then what are they doing with the time savings from llm. generating more remixing ? is there really so much demand for remix slop. i dont think so.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#179

Post by dominotw »

coldtea wrote: post #154
dominotw wrote: post #110

Earlier quoted context omitted.

[flagged]

I don't think you understand how LLMs work. They're not merely re-arranging pre-existing blocks of code. And they have been shown to develop emergent properties that weren't in their training set time and again. They generate novel things as much as the average programmer (which works after himself having practice, exposure to codebases, and training, and reading API documentation, and such) generates novel things.

since you understand how llms really work. show us what novel items llms have generated for you.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#180

Post by chanux »

sdevonoes wrote: post #138
guhcampos wrote: post #93

I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet. In my experience, for anything complex enough, I have to start adding more and more constraints,…

> At some point this starts to look like we're all just moving complexity from the more formal and deterministic world of programming languages to the informal and non-deterministic world of natural language. This is the problem nobody is talking about. I see codebases growing in MD files with instructions and guidelines and requests that are also LLM generated… and it’s all piling up. No one is reviewing it 100% , a…

Software loose on theory[1] trying to compensate with moar md.

[1] https://pages.cs.wisc.edu/~remzi/Naur.pdf

Post reply on HN