Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

arxiv.org

Post reply on HN
Print view View original

81–90 of 230 posts

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#81

Post by jumploops »

vishvananda wrote: post #41

I've been experimenting quite a bit with long-horizion agentic coding[1] and I have also noticed that agents seem to perform worse when forced into certain architectural patterns. I have found that is a bit better when including the constraints along the way instead of adding them after the fact. There seems to be a side-effect I have been calling "calcification", where a pattern starts appearing in the codebase and…

> agents seem to perform worse when forced into certain architectural patterns.

FWIW I've noticed this too. I've found that the agents/models have their own style, which is mostly summed up as overly verbose.

Additionally, the models are OK at modularization when given space to "plan" their implementation, but rarely decide that abstracting something would be helpful after the fact (i.e. after many iterations on a greenfield codebase or when being dropped into a legacy codebase).

This often leads to "god files" which, when pointed to by the user/architect, causes the models to correctly critique (humorously when they're the ones that wrote the code in the first place).

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#82

Post by lanstin »

skydhash wrote: post #80
Silhouette wrote: post #31

Earlier quoted context omitted.

If there is one good thing that the generative AI tools have shown beyond any doubt it's that the classic "good programming" practices are still useful and effective. Self-documenting code. Modular design. Clearly defined architecture. Incremental development. Coding standards. Automated tests. Automated everything . If there's a second thing the generative AI tools have shown beyond any doubt it's that many of the m…

> If there is one good thing that the generative AI tools have shown beyond any doubt it's that the classic "good programming" practices are still useful and effective If you apply those practice, then quickly you find yourself using the agent as merely a writing boost. And there’s an inflexion point when coding is no longer a bottleneck. Instead, you spend more time on thinking about design. You can see it in open s…

Yeah and that design and insight is the tiring part and while fun a bit less satisfying in the way that writing a nice bit of boiler plate or populating the struct members for your data type can be. One thing is you can work on design and insight while taking a good walk around the block, which is nice.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#83

Post by dominotw »

emp17344 wrote: post #11
maxbond wrote: post #3

Reminds me of the recent paper about delegating document editing tasks to LLMs across different disciplines [1]. That paper found that programming was the only discipline most LLMs can perform long horizon tasks on without accumulating errors & corrupting the document. I've only read the abstract of this one so far but it seems like this paper has zoomed in on programming with greater fidelity and shown a similar phe…

If it’s not easily verifiable, LLMs aren’t good at it.

but what does it mean to be good at something that cant be verified. how do you know that they are not good at it, you are obviously using some measure.

sounds like an oxymoron of a claim.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#84

Post by skydhash »

lanstin wrote: post #82
skydhash wrote: post #80

Earlier quoted context omitted.

> If there is one good thing that the generative AI tools have shown beyond any doubt it's that the classic "good programming" practices are still useful and effective If you apply those practice, then quickly you find yourself using the agent as merely a writing boost. And there’s an inflexion point when coding is no longer a bottleneck. Instead, you spend more time on thinking about design. You can see it in open s…

Yeah and that design and insight is the tiring part and while fun a bit less satisfying in the way that writing a nice bit of boiler plate or populating the struct members for your data type can be. One thing is you can work on design and insight while taking a good walk around the block, which is nice.

I spend that time mostly on the sofa, or in front of a whiteboard. Or sometimes a live brainstorming. Typing code is actually relaxing. What looks like relaxing is actually hard thinking.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#85

Post by dnautics »

withinboredom wrote: post #56
brandensilva wrote: post #39

Earlier quoted context omitted.

Right more simply put it's great at being a copy cat, exploring similar data points that match your token needs. It is not great at decision making or judgment calls that don't have a well defined spec or plan in place yet; like unofficial or unapproved tokens if you will. A lot of this stuff simply never has had specs as it has been internal to how companies work and their secret sauce. The closest thing we have are…

Doesn't that make sense? Its text prediction. If you give it examples, it can predict. Synthesizing "put semi-colons on new lines" requires it to generate its own examples 'in its head' (so to speak) and remember that. It won't. It's like when I see people feeding it a whole bunch of "best practices" and expect it to follow them. It won't. But you could ask it questions about the best practices all day long.

Supposing an unspecified or poorly specified function f(x), and example "f(A)=>B", "given C tell me what f(C) is" lies at the core of creativity.

Idk, calling it "just text prediction " seems unfairly dismissive of this capability

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#86

Post by alasano »

I've been building https://engine.build to introduce a proper structured external agent orchestrator that's used to build with clear constraints and make sure the end result is what you wrote in your spec or requirements. Without having to babysit and micromanage the models.

Implementation phases very often go through 5-10 review and fix rounds to actually get the implementation to match the spec. It takes longer but that's what's necessary to get actually good results on long horizon tasks with detailed requirements. I'll be open sourcing it fully soon.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#87

Post by Silhouette »

skydhash wrote: post #80
Silhouette wrote: post #31

Earlier quoted context omitted.

If there is one good thing that the generative AI tools have shown beyond any doubt it's that the classic "good programming" practices are still useful and effective. Self-documenting code. Modular design. Clearly defined architecture. Incremental development. Coding standards. Automated tests. Automated everything . If there's a second thing the generative AI tools have shown beyond any doubt it's that many of the m…

> If there is one good thing that the generative AI tools have shown beyond any doubt it's that the classic "good programming" practices are still useful and effective If you apply those practice, then quickly you find yourself using the agent as merely a writing boost. And there’s an inflexion point when coding is no longer a bottleneck. Instead, you spend more time on thinking about design. You can see it in open s…

If you apply those practice, then quickly you find yourself using the agent as merely a writing boost.

I don't know what that means but I have seen no evidence so far that if you don't apply those practices then your code will be anything other than unmanageable spaghetti if you leave AI to maintain it for long.

Coding has never been the bottleneck for good developers. Part of the reason for that is that good developers know how to isolate different aspects of a system and so keep each individual aspect relatively simple and self-contained. Another part is that good developers were already standardising and automating a lot of the grunt work. These traits are also advantageous for keeping generative AI on the right track and keeping its proposed changes manageable.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#89

Post by coredog64 »

qsort wrote: post #26
apsurd wrote: post #23

Earlier quoted context omitted.

Would you mind sharing antirez' suggestion?

I am obviously paraphrasing, but the general idea is that trying to synthesize style from a codebase into e.g. a markdown guide generally doesn't work very well. What achieves style transfer is providing the model with a lot of examples of the style, conventions, patterns you want. To put it in practice: if you point claude/codex to a repository and you ask it to implement feature X using style guide Y, the code will…

I was recently using Copilot to implement a small feature within a very large codebase. About 75-80% of the time, the code that was added matched the current style (warts and all). Copilot would specifically go off and research "How X is already done in the codebase" all the time.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#90

Post by maxbond »

dominotw wrote: post #83
emp17344 wrote: post #11

Earlier quoted context omitted.

If it’s not easily verifiable, LLMs aren’t good at it.

but what does it mean to be good at something that cant be verified. how do you know that they are not good at it, you are obviously using some measure. sounds like an oxymoron of a claim.

It means having taste. People say Picasso was a great painter, but that cannot be verified (at least, not in the sense of a verified reward).
Post reply on HN