Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

arxiv.org

Post reply on HN
Print view View original

51–60 of 230 posts

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#51

Post by BlueTierOps »

mikeyouse wrote: post #40
qsort wrote: post #26

Earlier quoted context omitted.

I am obviously paraphrasing, but the general idea is that trying to synthesize style from a codebase into e.g. a markdown guide generally doesn't work very well. What achieves style transfer is providing the model with a lot of examples of the style, conventions, patterns you want. To put it in practice: if you point claude/codex to a repository and you ask it to implement feature X using style guide Y, the code will…

I ran into similar issues as we started to roll out LLM generated financials in our org.. I’m so used to the old SQL workflow of “grab this data from this table, that data from that table, combine it into a final result that looks like xxxx” where the tables were outputs from reports in our ERP but I was having terrible results. Ended up pointing Claude at a few sample files from our existing reporting, gave it read-…

[flagged]

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#52

Post by [deleted] »

akomtu wrote: post #48
leecommamichael wrote: post #7

These things don’t think. We’re going to have to reiterate this for a long time, I fear.

There is a movie, Gold (2016), about a fake gold mine. One of its founders is a true believer: he found a few chunks of gold and started digging for more. The other founder is a nihilist: he realised that there is no gold there, but who cares if he makes the investors believe? So he does, and almost sells the company for $300M. In our story, investors are mining intelligence from GPUs, and they truly believe they are…

[deleted]

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#53

Post by UncleEntity »

nijave wrote: post #29
jdlshore wrote: post #4

“Our systematic study exposes a phenomenon of constraint decay in LLM-based coding agents. While current models excel at unconstrained generation, their performance drops when forced to navigate explicit architectural rules. For end-users, this dichotomy implies that agents are reliable for rapid prototyping but remain unreliable for production-grade backend development.” One major weakness of this study is that they…

Hmm, I have some anecdotal evidence this is true. Interactively working out a plan with Opus on multiple occasions it'd come up with an incompatible solution, I'll add additional context/requirements, and it has a tendency to "anchor" on it's original architecture and struggles to adapt. Sometimes it tries to sneak in changes for the original plan anyway.

I think the problem is they take the shortest path to the goal ...which may or may not coincide with what you have planned. Oh, and generally think instructions are merely suggestions and what you really want this this totally different thing and not the one in the plan you handed them plus, as a stoke of good luck, this other system is a lot easier to implement as well.

I mean, I spend more tokens having them clean up all the places they didn't follow the the plan (if I catch it) or implementing what came out of a 'complete and tested' previous plan where they just stop as soon as all the pathetic new test pass and you discover half of it isn't even there when trying to implement the next thing on top of it.

Though... I have been conducting an experiment, of sorts, where we've been cooking on these fairly complicated projects and I don't ever touch a single line of code, just yell at them a lot, and with suitable amounts of marijuana (they are very frustrating most of the time) it's been going pretty well. I also helps that they need to explain what they're doing to somebody fairly-baked -- maybe not such an HR friendly plan?

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#54

Post by KaiShips »

qsort wrote: post #26
apsurd wrote: post #23

Earlier quoted context omitted.

Would you mind sharing antirez' suggestion?

I am obviously paraphrasing, but the general idea is that trying to synthesize style from a codebase into e.g. a markdown guide generally doesn't work very well. What achieves style transfer is providing the model with a lot of examples of the style, conventions, patterns you want. To put it in practice: if you point claude/codex to a repository and you ask it to implement feature X using style guide Y, the code will…

[flagged]

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#55

Post by Animats »

jdlshore wrote: post #4

“Our systematic study exposes a phenomenon of constraint decay in LLM-based coding agents. While current models excel at unconstrained generation, their performance drops when forced to navigate explicit architectural rules. For end-users, this dichotomy implies that agents are reliable for rapid prototyping but remain unreliable for production-grade backend development.” One major weakness of this study is that they…

That may be the same problem seen when prompts try to force "alignment" or "guardrails". There's a performance drop. Seemingly, a big chunk of the potential solution space has been made unreachable.

For example, if you apply "guardrails" to an image generator of about a year ago, all the people start looking alike. Story generators start using only a few standard names.

That was last year. Is it happening with the frontier models?

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#56

Post by withinboredom »

brandensilva wrote: post #39
qsort wrote: post #26

Earlier quoted context omitted.

I am obviously paraphrasing, but the general idea is that trying to synthesize style from a codebase into e.g. a markdown guide generally doesn't work very well. What achieves style transfer is providing the model with a lot of examples of the style, conventions, patterns you want. To put it in practice: if you point claude/codex to a repository and you ask it to implement feature X using style guide Y, the code will…

Right more simply put it's great at being a copy cat, exploring similar data points that match your token needs. It is not great at decision making or judgment calls that don't have a well defined spec or plan in place yet; like unofficial or unapproved tokens if you will. A lot of this stuff simply never has had specs as it has been internal to how companies work and their secret sauce. The closest thing we have are…

Doesn't that make sense? Its text prediction. If you give it examples, it can predict. Synthesizing "put semi-colons on new lines" requires it to generate its own examples 'in its head' (so to speak) and remember that. It won't.

It's like when I see people feeding it a whole bunch of "best practices" and expect it to follow them. It won't. But you could ask it questions about the best practices all day long.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#58

Post by pianopatrick »

I think someone is going to figure out a framework for using LLMs for coding.

A framework would use static code checking tools to force an architecture on to LLMs instead of trying to do so in markdown.

I don't know exactly what it will look like but for example I could imagine a Java Framework where the LLM could only create subclasses of certain classes.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#59

Post by AmazingTurtle »

So my finding is: planning is worth it.

For a little complex changes, I always run codex (5.5-high) in planning mode first. I have linked various docs/{ARCHITECTURE,BACKEND-GUIDELINES,NESTJS-DI,..}.md etc. from AGENTS.md so they can quickly discover relevant docs at planning time, only if they are needed. No need to know react specific stuff when it's dealing with a backend problem for example. I typically blindly approve plans made by the agent with a fresh context, because that's as if I had prompted it. Works the best for me.

Using /goal however, it's really just constantly compacting and doing it's thing, of course it gets sloppy. If only there was a state machine that would transform tickets into a Planning Mode Prompt, then use, idk. guardian approvals (somehow a "Product Management Perspective Lens" approving or making changes to the plan) and then letting a less capable or less reasoning agent execute the plan, I think that would work the best.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#60

Post by UncleEntity »

Npovview wrote: post #33
noosphr wrote: post #13

Earlier quoted context omitted.

To this day frontier models think that A and not B means A and B when the sentence gets pushed far enough back in their context window. The context length that model can reason over without obvious errors is much smaller than the advertised context. Between a 1/4th to a 1/20th what is advertised on the tin.

Do you also happen to remember what you ate last thrusday?

"If you have a question look in the specification for the answer and don't just guess" seems a fairly important thing to remember for more than a couple of minutes...
Post reply on HN