Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

arxiv.org

Post reply on HN
Print view View original

91–100 of 230 posts

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#91

Post by 8note »

qsort wrote: post #26
apsurd wrote: post #23

Earlier quoted context omitted.

Would you mind sharing antirez' suggestion?

I am obviously paraphrasing, but the general idea is that trying to synthesize style from a codebase into e.g. a markdown guide generally doesn't work very well. What achieves style transfer is providing the model with a lot of examples of the style, conventions, patterns you want. To put it in practice: if you point claude/codex to a repository and you ask it to implement feature X using style guide Y, the code will…

You basically get this for free, if the coding agent has read the relevant classes that the legacy code its touching has to match.

just dont break out a plan without also having it read the code again

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#92

Post by withinboredom »

dnautics wrote: post #85
withinboredom wrote: post #56

Earlier quoted context omitted.

Doesn't that make sense? Its text prediction. If you give it examples, it can predict. Synthesizing "put semi-colons on new lines" requires it to generate its own examples 'in its head' (so to speak) and remember that. It won't. It's like when I see people feeding it a whole bunch of "best practices" and expect it to follow them. It won't. But you could ask it questions about the best practices all day long.

Supposing an unspecified or poorly specified function f(x), and example "f(A)=>B", "given C tell me what f(C) is" lies at the core of creativity. Idk, calling it "just text prediction " seems unfairly dismissive of this capability

Saying that it’s dismissive is like saying writing (insert language) is dismissive that you’re just writing assembly.

at the end of the day, it presents a vector field and predicts the next vector. That’s literally the heart of intelligence just like assembly is the heart of execution. When playing table tennis, your brain is literally predicting seconds into the future to get your body into the right position.

But we aren’t discussing intelligence here. We are discussing how best to utilize that intelligence.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#93

Post by guhcampos »

I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated.

That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet.

In my experience, for anything complex enough, I have to start adding more and more constraints, style guides, corner cases, error handling, optimization guidelines and all this good stuff to my Markdown specifications, rules and skills. At some point this starts to look like we're all just moving complexity from the more formal and deterministic world of programming languages to the informal and non-deterministic world of natural language. The writing speed gains are enormous, yeah, and business sees this as productivity gains, of course - and we do it because the pressure for increased productivity is there, as it's always been; yet the trade off seems to be clear and a lot of people are just ignoring it.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#94

Post by alexwwang »

I am trying to avoid this by building a plugin based on my memory management project Aristotle. I add a status machine to monitor the activities of LLM while it does jobs following my tdd-pipeline skills, which begins with requirements clarification and ends up with delivery.

These two projects are on GitHub, you may search alexwwang/aristotle and alexwwang/tdd-pipeline to dive into the details or just ask your LLM to scan them to tell you the points you are interested in.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#96

Post by dnautics »

withinboredom wrote: post #92
dnautics wrote: post #85

Earlier quoted context omitted.

Supposing an unspecified or poorly specified function f(x), and example "f(A)=>B", "given C tell me what f(C) is" lies at the core of creativity. Idk, calling it "just text prediction " seems unfairly dismissive of this capability

Saying that it’s dismissive is like saying writing (insert language) is dismissive that you’re just writing assembly. at the end of the day, it presents a vector field and predicts the next vector. That’s literally the heart of intelligence just like assembly is the heart of execution. When playing table tennis, your brain is literally predicting seconds into the future to get your body into the right position. But w…

You're making my point for me, saying table tennis is "just a proprioceptive predictor" is dismissively reductive (and not a particularly useful framework for understanding table tennis), even if it is strictly speaking accurate. It's the sort of thing someone who has no idea how hard training for table tennis is would say.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#97

Post by dominotw »

maxbond wrote: post #90
dominotw wrote: post #83

Earlier quoted context omitted.

but what does it mean to be good at something that cant be verified. how do you know that they are not good at it, you are obviously using some measure. sounds like an oxymoron of a claim.

It means having taste. People say Picasso was a great painter, but that cannot be verified (at least, not in the sense of a verified reward).

"people say picasso was a great painter" is definitely not hard to verify . lol.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#98

Post by [deleted] »

dominotw wrote: post #97
maxbond wrote: post #90

Earlier quoted context omitted.

It means having taste. People say Picasso was a great painter, but that cannot be verified (at least, not in the sense of a verified reward).

"people say picasso was a great painter" is definitely not hard to verify . lol.

[deleted]

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#100

Post by ipaddr »

nozzlegear wrote: post #72
jeremyjh wrote: post #24

Earlier quoted context omitted.

Even the strongest frontier model they used - GPT 5.2 - I would consider barely usable for agentic programming. I’m not really interested in analysis of the weaknesses of such models because in my experience many weaknesses disappear entirely as models get stronger and reasoning effort is turned up. Especially if you tell them what you want them to do. Also, it’s not surprising to learn that when more acceptance crit…

Oldheads remember when GPT 5.2 was at the forefront of agentic programming. December 2025 feels like eons ago, but alack it was an entire half year!

If I'm not using got 5.5 high reasoning I'm wasting time.
Post reply on HN