Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

arxiv.org

Post reply on HN
Print view View original

71–80 of 230 posts

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#71

Post by zdragnar »

piker wrote: post #64
zdragnar wrote: post #62

Earlier quoted context omitted.

I've noticed something similar with AI assist authored books as well. Early on it does alright, but after some chapters the beginning of each chapter repeats the end of the previous, and obvious LLM tells become more frequent. The more it has to go on, the more it relies on repetition of what came before. It's also possible that authors start paying much less attention and put less effort into editing later chapters.…

Holy crap are you reading books that advertised somehow they were written with LLM assistance? Hard no here in 2026.

Oh no, they were not advertised as such. It's rather painfully obvious in the worst cases.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#72

Post by nozzlegear »

jeremyjh wrote: post #24
jdlshore wrote: post #4

“Our systematic study exposes a phenomenon of constraint decay in LLM-based coding agents. While current models excel at unconstrained generation, their performance drops when forced to navigate explicit architectural rules. For end-users, this dichotomy implies that agents are reliable for rapid prototyping but remain unreliable for production-grade backend development.” One major weakness of this study is that they…

Even the strongest frontier model they used - GPT 5.2 - I would consider barely usable for agentic programming. I’m not really interested in analysis of the weaknesses of such models because in my experience many weaknesses disappear entirely as models get stronger and reasoning effort is turned up. Especially if you tell them what you want them to do. Also, it’s not surprising to learn that when more acceptance crit…

Oldheads remember when GPT 5.2 was at the forefront of agentic programming. December 2025 feels like eons ago, but alack it was an entire half year!

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#74

Post by shepherdjerred »

mrob wrote: post #44
acbart wrote: post #14

Earlier quoted context omitted.

It's crazy to me that people think of Python as dynamically typed by default. Strong static typing has been an option in Python for years now, and it should just be the default.

>Strong static typing has been an option in Python for years now, and it should just be the default. https://docs.python.org/3/library/typing.html "The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc." Which third-party enforcement mechanism do you propose become the default?

There are plenty of options for static type checking in Python. Choose your favorite or just use Ty

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#75

Post by shepherdjerred »

epgui wrote: post #16
acbart wrote: post #14

Earlier quoted context omitted.

It's crazy to me that people think of Python as dynamically typed by default. Strong static typing has been an option in Python for years now, and it should just be the default.

The python type hints are useful for static analysis (and yes, should be the default) but it’s a joke compared to the utility of types in a language like Haskell.

If you're comparing type systems against Haskell you're excluding all mainstream languages except maybe Scala and Rust

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#76

Post by shepherdjerred »

antonvs wrote: post #49
acbart wrote: post #14

Earlier quoted context omitted.

It's crazy to me that people think of Python as dynamically typed by default. Strong static typing has been an option in Python for years now, and it should just be the default.

Typing with tools like Pyright doesn't come close to providing what a good statically typechecked language provides. There are many reasons for this. A big one is that many libraries are only partially typed at best, and dynamic types tend to propagate, weakening the guarantees you get from type checking. Dynamic idioms in general, including something as common as string-indexed dictionaries, negate type checking. Ru…

TypeScript had this _exact_ same problem when it started out. As more libraries add annotations, the ecosystem will become stronger, and it will eventually be about as good as a "real" statically typed language.

> Dynamic idioms in general, including something as common as string-indexed dictionaries, negate type checking.

Do you have any proof of this? It hasn't been a problem in TypeScript, and I doubt it's an issue in Python

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#77

Post by dalemhurley »

This is why we as an industry have spent so much effort optimising the code generation process with things like skills, rules, tests, reviews, lints, agentic loops with feedback and sub-agents, and the code-runners. It is not just LLMs building code, it is an eco-system collaborating together.

I would agree too that as the codebase grows the LLM struggles more and more with generating code. It is probably misaligned incentives, it wants to complete the isolated task without too much context consumed, at the POC it can consume most of the app, by about 30K lines of code it is quite complex code base to navigate.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#78

Post by noosphr »

antonvs wrote: post #50
noosphr wrote: post #13

Earlier quoted context omitted.

To this day frontier models think that A and not B means A and B when the sentence gets pushed far enough back in their context window. The context length that model can reason over without obvious errors is much smaller than the advertised context. Between a 1/4th to a 1/20th what is advertised on the tin.

Critiques like this tend to focus very hard on what models can't do. It's true, they have limitations. But they're also superhuman in so many other ways. It's valid to point out limitations, but that doesn't support the conclusion that models are not incredibly powerful and capable of the functional equivalent of reasoning at human or superhuman levels in many scenarios.

They may be better than humans at reasoning but they are substantially worse than the first generation logic programs from the 1950s.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#80

Post by skydhash »

Silhouette wrote: post #31
lanstin wrote: post #21

Earlier quoted context omitted.

And doing incremental dev, so once a feature is done you can mostly ignore it.

If there is one good thing that the generative AI tools have shown beyond any doubt it's that the classic "good programming" practices are still useful and effective. Self-documenting code. Modular design. Clearly defined architecture. Incremental development. Coding standards. Automated tests. Automated everything . If there's a second thing the generative AI tools have shown beyond any doubt it's that many of the m…

> If there is one good thing that the generative AI tools have shown beyond any doubt it's that the classic "good programming" practices are still useful and effective

If you apply those practice, then quickly you find yourself using the agent as merely a writing boost. And there’s an inflexion point when coding is no longer a bottleneck. Instead, you spend more time on thinking about design. You can see it in open source projects where most PRs are just a few line diffs. The bottleneck is knowledge and problem solving talent.

Post reply on HN