Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

arxiv.org

Post reply on HN
Print view View original

191–200 of 230 posts

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#191

Post by leecommamichael »

antonvs wrote: post #50
noosphr wrote: post #13

Earlier quoted context omitted.

To this day frontier models think that A and not B means A and B when the sentence gets pushed far enough back in their context window. The context length that model can reason over without obvious errors is much smaller than the advertised context. Between a 1/4th to a 1/20th what is advertised on the tin.

Critiques like this tend to focus very hard on what models can't do. It's true, they have limitations. But they're also superhuman in so many other ways. It's valid to point out limitations, but that doesn't support the conclusion that models are not incredibly powerful and capable of the functional equivalent of reasoning at human or superhuman levels in many scenarios.

Agreed. Both are true. I sometimes think of the calculator as being superhuman as well.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#192

Post by dominotw »

tpmoney wrote: post #189
dominotw wrote: post #178

Earlier quoted context omitted.

then what are they doing with the time savings from llm. generating more remixing ? is there really so much demand for remix slop. i dont think so.

I don't think I've ever worked on a project where there wasn't more work to be done than there was time to do it in.

ai can remix things on the fly so lots of widgets that were being stamped on are not needed anymore.

we are in middle state where ai tools to generate on the fly widget work arent accessible in the form that most ppl need. So programmers are currently doing the manual step by managing remix into easily consumable form.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#193

Post by rodphil »

bitexploder wrote: post #190
tpmoney wrote: post #188

Earlier quoted context omitted.

One question I have is are these "constraints, style guides, corner cases, error handling, optimization guidelines" extra things that you wouldn't need otherwise, or are they formal documentation of the baked in assumptions and knowledge accumulated over the years? Every project I've ever worked on has had heaps of shared knowledge that's just part of stuff the team just "knows" and no one ever really writes down. Th…

There is a great thing. Because the agents can do so much toil you can add things like formal verification, fuzzing, and other feedback mechanisms and quality gates to your projects cheaply. In a human written project you still needed those things, but it cost a lot. Agents require these quality gates and they can implement them for you. The problem with AI documentation is it will just write a lot of useless bullshi…

Yeah "plus one" to this. Static analysis, fuzzing, linting, integration tests -- there are all sorts of very useful artifacts which have been around for a long time, but which are very time consuming to implement and then maintain. LLMs shift the economics around producing and maintaining these tremendously, so we can now afford these robust validation mechanisms.

These serve as living documentation which cries out in pain when they get out of sync with the system in question, generating specific error messages -- as opposed to natural language docs which rapidly drift into an ambiguous "kinda useful" state. And the validation is performed mechanically (as opposed to neurally) so no hallucinations are possible.

The one thing I would add is that you do want these artifacts to be human-friendly from a reading perspective -- you want engineers to be able to scan over these and check that they are validating the right things.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#194

Post by epgui »

shepherdjerred wrote: post #75
epgui wrote: post #16

Earlier quoted context omitted.

The python type hints are useful for static analysis (and yes, should be the default) but it’s a joke compared to the utility of types in a language like Haskell.

If you're comparing type systems against Haskell you're excluding all mainstream languages except maybe Scala and Rust

Yes.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#195

Post by ipaddr »

nozzlegear wrote: post #104
ipaddr wrote: post #100

Earlier quoted context omitted.

If I'm not using got 5.5 high reasoning I'm wasting time.

Well, maybe so, but how did you feel about 5.2 when it was OpenAI's frontier model? That's what I'm getting at – it was the equivalent of your gpt 5.5 high reasoning just six months ago.

It was a joke. I think you need to mix up models.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#196

Post by ethin »

Npovview wrote: post #164
ethin wrote: post #102

Earlier quoted context omitted.

Do you have a point? Because last time I checked, AIs were supposed to be better than us fragile faulty humans, and weren't designed to emulate us and all our faults.

If you have been following the news, harness is also a scaling direction now. Prompt your AI better not to forget relevant stuff or write them in a file which it can refer later. This way context can be refreshed, this is cached facts method or rolling window method of refreshing your memory just like you would ask a colleague to explain a concept again. These are solved problems.

Are they though? Because I really shouldn't have to use Claude Code (and I don't) just to get even decent results. As I said, I thought one of the biggest advantages AI was supposed to have was that it wouldn't need such constant reminding of things because it wasn't trying to emulate us faulty, forgetful, fragile humans who do have memory loss?

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#197

Post by jeremyjh »

emp17344 wrote: post #47
jeremyjh wrote: post #18

Earlier quoted context omitted.

I think that’s mostly because they get so much more of that reinforcement learning - since it is so economical. I dont know if there is any evidence of a fundamental reason they can’t be just as good at other tasks, but it might be economically infeasible for awhile yet.

RLVR doesn’t work for unverifiable tasks, so they won’t be able to effectively use tools to boost reliability for those tasks.

Right, so you have to use RLHF. That is the economics problem I was referring to.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#198

Post by jeremyjh »

mjburgess wrote: post #28
jeremyjh wrote: post #18

Earlier quoted context omitted.

I think that’s mostly because they get so much more of that reinforcement learning - since it is so economical. I dont know if there is any evidence of a fundamental reason they can’t be just as good at other tasks, but it might be economically infeasible for awhile yet.

No one is curating vast amounts of data for them in other domains. Programmers send programs with fixes

Its more about how costly it is to verify work in reinforcement learning. It is cheap in Mathematics and coding because it can be automated. It is expensive in other domains because while you can capture certain datasets to do pre-training on, you ultimately need humans in the loop to judge the quality of work.

Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation

#200

Post by maxbond »

dominotw wrote: post #181
Geezus_42 wrote: post #146

Earlier quoted context omitted.

There is a huge difference between "not verifiable" and "not easily verifiable".

No because if op is actually able to verify it ( with difficulty) then ai can do it too.

No one in this thread appears to disagree. The issue is that RLHF is prohibitively expensive and the number of disciplines you could target is massive, so for reasons of economics rather than fundamental theory, AIs do not perform well on tasks that aren't amenable to RLVR and even then off the shelf LLMs are really only well aligned for programming.

In the paper I linked they created a benchmark spanning 80 disciplines with tasks that could be checked automatically. So these are necessarily tasks that are tractable for RLVR, trivially you could use performance against the benchmark as a reward function. The performance was still mediocre in everything but programming. And as we're seeing in this article, there is still room for growth in programming.

In general you seem to be reading very literally in some places (taking the statement "AIs aren't good at X" as applying to all AI and perpetually) and very loosely in others (disregarding "easily" as unimportant) and misinterpreting statements you appear to agree with as being in disagreement. I don't think there's a real disagreement here, I think there's a misunderstanding.

Post reply on HN