[flagged]
Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
111–120 of 230 posts
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#112I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet. In my experience, for anything complex enough, I have to start adding more and more constraints,…
[flagged]
LLMs recently solved a major, famous open mathematical problem in combinatorial geometry:
https://www.reddit.com/r/math/comments/1tj534d/openais_inter...
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#113I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet. In my experience, for anything complex enough, I have to start adding more and more constraints,…
[flagged]
There is nothing new under the sun.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#114I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet. In my experience, for anything complex enough, I have to start adding more and more constraints,…
[flagged]
And how much of pre-LLM code was just copy pasta from Stack Overflow?
Code doesn't need to be novel to be useful. There's a reason why design patterns are a thing in software.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#115Earlier quoted context omitted.
Even the strongest frontier model they used - GPT 5.2 - I would consider barely usable for agentic programming. I’m not really interested in analysis of the weaknesses of such models because in my experience many weaknesses disappear entirely as models get stronger and reasoning effort is turned up. Especially if you tell them what you want them to do. Also, it’s not surprising to learn that when more acceptance crit…
Wait isn't gpt 5.2 good? Or is it not thinking / not codex? 5.2 was what sparked the late 2025 openai agentic programming revolution.
5.2 still had a Codex variant, which this doesn't describe using. It also notably is not using the Codex harness -- it does everything with open source harnesses (which obviously are worse). And while it uses two harnesses with its cheap models, it only uses the worse-performing one of those with GPT 5.2 for cost reasons. (They also don't specify effort/thinking level used for GPT 5.2, but given that it performs worse in their baseline testing than obviously non-SOTA models, I'm guessing it wasn't set to anything high.)
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#116Earlier quoted context omitted.
I don't know if you're being factitious or not but that was not what I meant. Picasso being a great painter is an example of "having taste"; "create an artistic image generation model with Picasso-level performance" is a valid problem statement we could attack with RLHF, but not with RLVR, because "taste" is not amenable to modeling with a reward function. "Write this code in a way that is readable and maintainable"…
https://futurism.com/artificial-intelligence/real-monet-ai-c...
[deleted]
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#117Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#118Earlier quoted context omitted.
[flagged]
And how much of pre-LLM code was just copy pasta from Stack Overflow? Code doesn't need to be novel to be useful. There's a reason why design patterns are a thing in software.
That’s why we abstract the useful code away as libraries, frameworks, etc.
AI is not an abstraction.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#119Earlier quoted context omitted.
And how much of pre-LLM code was just copy pasta from Stack Overflow? Code doesn't need to be novel to be useful. There's a reason why design patterns are a thing in software.
That’s why we abstract the useful code away as libraries, frameworks, etc. AI is not an abstraction.
You generally need to wire libraries in to your service, and you may be using the library in a slightly different way than normal. AIs are perfectly capable of doing this.
Back to the original point, though: most software engineering work isn't novel. Most people are working on slightly different iterations of the same thing, but with the aim of different products. You can have completely different products that use nearly the same patterns as most other services.
To put it bluntly: we don't need AI to generate novel code for the vast majority of the software being built.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#120[flagged]