Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

arxiv.org

Post reply on HN
Print view View original

211–220 of 230 posts

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#211

Post by hansmayer »

coldtea wrote: post #154
dominotw wrote: post #110

Earlier quoted context omitted.

[flagged]

I don't think you understand how LLMs work. They're not merely re-arranging pre-existing blocks of code. And they have been shown to develop emergent properties that weren't in their training set time and again. They generate novel things as much as the average programmer (which works after himself having practice, exposure to codebases, and training, and reading API documentation, and such) generates novel things.

> And they have been shown to develop emergent properties that weren't in their training set time and again.

Ah yes, the famous emergent properties - like suggesting that we should walk to the car wash?

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#212

Post by hansmayer »

coldtea wrote: post #202
dominotw wrote: post #179

Earlier quoted context omitted.

since you understand how llms really work. show us what novel items llms have generated for you.

How about a novel theorem escaping mathematicians for close to a century? https://www.scientificamerican.com/article/ai-just-solved-an...

The question was, what did they solve FOR YOU.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#213

Post by nozzlegear »

ipaddr wrote: post #195
nozzlegear wrote: post #104

Earlier quoted context omitted.

Well, maybe so, but how did you feel about 5.2 when it was OpenAI's frontier model? That's what I'm getting at – it was the equivalent of your gpt 5.5 high reasoning just six months ago.

It was a joke. I think you need to mix up models.

Gotcha. Hard to parse tone and intent through text on the internet.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#214

Post by tcoff91 »

hansmayer wrote: post #208
tcoff91 wrote: post #140

Earlier quoted context omitted.

This is why you need to be generating more linter rules instead of just having things be in markdown files. I had never written an eslint rule until i started having agents pump them out for me and now I've encoded a bunch of important rules as lint rules that will fail CI if violated.

A linter won't prevent your idiot LLM from going bonkers and suddenly switching to GQL instead of REST just for that one endpoint, because it confabulated something or putting your stripe secret into your react frontend - all cases of slop I've seen happen.

That's why we still do code review. The linter rules is just about lowering the amount of mistakes you have to catch at code review time.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#216

Post by __mharrison__ »

dominotw wrote: post #178
__mharrison__ wrote: post #172

Earlier quoted context omitted.

99% of coders don't need to generate anything novel.

then what are they doing with the time savings from llm. generating more remixing ? is there really so much demand for remix slop. i dont think so.

My job is different from most developers these days.

But I'm writing more code than I ever did as a developer. YMMV

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#217

Post by bitexploder »

hansmayer wrote: post #209
bitexploder wrote: post #190

Earlier quoted context omitted.

There is a great thing. Because the agents can do so much toil you can add things like formal verification, fuzzing, and other feedback mechanisms and quality gates to your projects cheaply. In a human written project you still needed those things, but it cost a lot. Agents require these quality gates and they can implement them for you. The problem with AI documentation is it will just write a lot of useless bullshi…

> Because the agents can do so much toil you can add things like formal verification, fuzzing, and other feedback mechanisms and quality gates to your projects cheaply Works great until they sweep you a test under the rug which always passes because the condition is something like if(true) .

That was my point. Validating actual behavioral tests. Not letting them cheat. They still will at times, but like, resd their code, fix it or send a reviewer agent to find and make todo list. If you give them a behavioral test skill it will do a much better job. Sometimes I have to hint to them. I rarely ship anything I have not reviewed at least once.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#218

Post by suttontom »

coldtea wrote: post #153
sdevonoes wrote: post #139

Earlier quoted context omitted.

I don’t think that’s true based on experience. Maybe “<“ instead of “<<“, yeah. But even in that case, it’s an awful trade off for any serious codebase that needs to be maintained over the years (and you don’t know what LLMs are gonna look like next year, so there are zero guarantees all your MD is gonna work as good as it’s “working” right now)

As long as LLMs remains at the same skill level at coding, or better, there's 100% guarantee an MD (a glorified prompt) is gonna work as good as it’s “working” right now.

Models are not innately backwards-compatible. Both OpenAI and Anthropic encourage running evaluations and comparing the performance of your existing agent workflows against new models before just stepping up to the newest one because you may encounter regressions. I myself have seen lengthy/long-horizon multi-agent workflows begin breaking after moving to a newer model because for some reason the prompt containing an instruction to call a tool that worked 99/100 times before suddenly just stops working and needs to be modified.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#220

Post by hansmayer »

bitexploder wrote: post #217
hansmayer wrote: post #209

Earlier quoted context omitted.

> Because the agents can do so much toil you can add things like formal verification, fuzzing, and other feedback mechanisms and quality gates to your projects cheaply Works great until they sweep you a test under the rug which always passes because the condition is something like if(true) .

That was my point. Validating actual behavioral tests. Not letting them cheat. They still will at times, but like, resd their code, fix it or send a reviewer agent to find and make todo list. If you give them a behavioral test skill it will do a much better job. Sometimes I have to hint to them. I rarely ship anything I have not reviewed at least once.

> Not letting them cheat. They still will at times, but like, resd their code,

Well then, if they "still will", your effort kind of misses the point. Sure maybe, you'll catch it every time and maybe that one time you did not catch it, it was no critical mistake...But it only needs to make that critical mistake once, and all of this effort was in vain.

Post reply on HN