Earlier quoted context omitted.
If I'm not using got 5.5 high reasoning I'm wasting time.
Well, maybe so, but how did you feel about 5.2 when it was OpenAI's frontier model? That's what I'm getting at – it was the equivalent of your gpt 5.5 high reasoning just six months ago.
Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
151–160 of 230 posts
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#152Earlier quoted context omitted.
> Back to LoC/s as a measure of "productivity." IMO this doesn’t follow from what OP wrote. I personally measure it with a more abstract “how long does it take me to ship something that is useful in production and solving a real problem” and the increase in speed there has been massive for me. But of course I’m not a bigbrain 10x coder that is doing bleeding edge novel stuff like most people here, so gains might be m…
> how long does it take me to ship something that is useful in production and solving a real problem But that’s only half of the problem. What about “and how easy it is to maintain long-term”. If you say that maintenance can be done via LLM, I would argue that there is zero guarantees that LLMs are backwards compatible and that the markdown you wrote now will work just as fine in 1,2,3 years
That this would be the case is even more guaranteed than some programming language being backwards compatible and the code we wrote working just as fine in 1,2,3, years.
Languages do get non-backwards compatible changes, dependencies break, stuff is deprecated, etc.
But the job of LLMs will remain to generate something from a prompt, and the markdown we wrote, as it's high level and not tied to language versions, APIs, and implementation details, will be just as good a prompt for that in 2050 as it is in 2026.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#153Earlier quoted context omitted.
> yet the trade off seems to be clear and a lot of people are just ignoring it. There's plenty of focus on the negative side of the tradeoff. Less so on why we're making it anyway, or why it somehow works out even if "this starts to look like we're all just moving complexity from the more formal and deterministic world of programming languages to the informal and non-deterministic world of natural language". And the…
I don’t think that’s true based on experience. Maybe “<“ instead of “<<“, yeah. But even in that case, it’s an awful trade off for any serious codebase that needs to be maintained over the years (and you don’t know what LLMs are gonna look like next year, so there are zero guarantees all your MD is gonna work as good as it’s “working” right now)
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#154I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet. In my experience, for anything complex enough, I have to start adding more and more constraints,…
[flagged]
They're not merely re-arranging pre-existing blocks of code.
And they have been shown to develop emergent properties that weren't in their training set time and again.
They generate novel things as much as the average programmer (which works after himself having practice, exposure to codebases, and training, and reading API documentation, and such) generates novel things.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#155Earlier quoted context omitted.
And how much of pre-LLM code was just copy pasta from Stack Overflow? Code doesn't need to be novel to be useful. There's a reason why design patterns are a thing in software.
That’s why we abstract the useful code away as libraries, frameworks, etc. AI is not an abstraction.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#156I've been building https://engine.build to introduce a proper structured external agent orchestrator that's used to build with clear constraints and make sure the end result is what you wrote in your spec or requirements. Without having to babysit and micromanage the models. Implementation phases very often go through 5-10 review and fix rounds to actually get the implementation to match the spec. It takes longer but…
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#157Earlier quoted context omitted.
There is nothing new under the sun.
Everything changes, nothing remains without change.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#158I think someone is going to figure out a framework for using LLMs for coding. A framework would use static code checking tools to force an architecture on to LLMs instead of trying to do so in markdown. I don't know exactly what it will look like but for example I could imagine a Java Framework where the LLM could only create subclasses of certain classes.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#159This is why we as an industry have spent so much effort optimising the code generation process with things like skills, rules, tests, reviews, lints, agentic loops with feedback and sub-agents, and the code-runners. It is not just LLMs building code, it is an eco-system collaborating together. I would agree too that as the codebase grows the LLM struggles more and more with generating code. It is probably misaligned…
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#160I've been building https://engine.build to introduce a proper structured external agent orchestrator that's used to build with clear constraints and make sure the end result is what you wrote in your spec or requirements. Without having to babysit and micromanage the models. Implementation phases very often go through 5-10 review and fix rounds to actually get the implementation to match the spec. It takes longer but…
Why link to an empty GitHub on your site?
If you don't want to sign up to be notified by email you can watch the org.