Earlier quoted context omitted.
doesnt make a difference to my comment
There is a huge difference between "not verifiable" and "not easily verifiable".
No because if op is actually able to verify it ( with difficulty) then ai can do it too.
Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
181–190 of 230 posts
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#182Earlier quoted context omitted.
I don’t think that’s true based on experience. Maybe “<“ instead of “<<“, yeah. But even in that case, it’s an awful trade off for any serious codebase that needs to be maintained over the years (and you don’t know what LLMs are gonna look like next year, so there are zero guarantees all your MD is gonna work as good as it’s “working” right now)
As long as LLMs remains at the same skill level at coding, or better, there's 100% guarantee an MD (a glorified prompt) is gonna work as good as it’s “working” right now.
This is quite a claim without any evidence to substantiate it. LLMs are nondeterministic models, whose behaviour is reliant on training data, model architecture and context (both in the general and domain specific sense).
There is absolutely no guarantee llm1(MD) == llm2(MD), by design. With the current batch you need to explicitly constrain a number of parameters, far more than simply the prompt, to get identical output from the _same_ model, let alone another model that has varied training data and/or architecture.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#183Earlier quoted context omitted.
Not to be pedantic but the emergent properties are in the training set, and thus the model and algorithm. There's no magic coming from the universe. What makes the behavior emergent is that it can't be predicted at training time. The emergent and unpredictable output is the result of massive vector complexity being encoded.
> Not to be pedantic but the emergent properties are in the training set, and thus the model and algorithm. There's no magic coming from the universe. You are either being pedantic or missing the point of emergent however. Yes, it's not some novel unforeseen thing, like a magical Marvel Universe material or some unknown to humanity mode of thinking. Same way when people make something new they still recombine known w…
My point was that if training data + encoding/training = model with emergent behaviour
The emergent behaviour is in the training data and/or encoding/training.
So while I agree it is emergent from the complexity, it isn't some unknown mechanism. Just complexity at scale.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#184[flagged]
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#185Earlier quoted context omitted.
This is why you need to be generating more linter rules instead of just having things be in markdown files. I had never written an eslint rule until i started having agents pump them out for me and now I've encoded a bunch of important rules as lint rules that will fail CI if violated.
Who lints the linters
Linter linters, obviously
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#186[flagged]
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#187[dead]
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#188I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet. In my experience, for anything complex enough, I have to start adding more and more constraints,…
One question I have is are these "constraints, style guides, corner cases, error handling, optimization guidelines" extra things that you wouldn't need otherwise, or are they formal documentation of the baked in assumptions and knowledge accumulated over the years? Every project I've ever worked on has had heaps of shared knowledge that's just part of stuff the team just "knows" and no one ever really writes down. Things like "sure you can use java's built in assert for tests, but we don't compile or run the application with the flags that enable them. Use junit's assertions/use the assertj library." or "prefer using auto generated accessors instead of manually writing them out". Even things like "if you change the structure of this ID string, you need to change all the code in modules A, B, and C because they all rely on the ID being in a certain format".
If you're really lucky, maybe a lot of this is documented in some wiki page somewhere, but everyone knows the documentation is never as complete as you'd like it to be. The longer a team works together without new people coming on board, the more likely it is that the documentation of these soft requirements and knowledge has drifted from reality. IME nothing shows how much you've failed to document than revisiting your onboarding process documents for the first time 2-3 years after you wrote them.
As I've experimented with the various AI tools, I feel like a lot of these extra documents I've written are documenting a lot of these things "everyone knows". But I'm also not at the "80% of the professional code I write is generated" stage yet. So I'm curious if you're finding that you're creating documentation that goes beyond just documenting what we used to just keep in our heads and are now getting into "writing a book about how to code" territory?
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#189Earlier quoted context omitted.
99% of coders don't need to generate anything novel.
then what are they doing with the time savings from llm. generating more remixing ? is there really so much demand for remix slop. i dont think so.
I don't think I've ever worked on a project where there wasn't more work to be done than there was time to do it in.
Re: Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
#190I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet. In my experience, for anything complex enough, I have to start adding more and more constraints,…
One question I have is are these "constraints, style guides, corner cases, error handling, optimization guidelines" extra things that you wouldn't need otherwise, or are they formal documentation of the baked in assumptions and knowledge accumulated over the years? Every project I've ever worked on has had heaps of shared knowledge that's just part of stuff the team just "knows" and no one ever really writes down. Th…
There is a great thing. Because the agents can do so much toil you can add things like formal verification, fuzzing, and other feedback mechanisms and quality gates to your projects cheaply. In a human written project you still needed those things, but it cost a lot. Agents require these quality gates and they can implement them for you. The problem with AI documentation is it will just write a lot of useless bullshit unless you guide it on what is important. You can also get agents to identify transitive dependencies via testing and other things.
I adopt the mindset of docs are for humans, tests are for agents. They document formal dependencies and leave a measurable artifact behind. If you identify some behavior or transitive dep in your system, agents document it first with a test codifying the expected behavior. Tests are the source of truth about expected system behavior and you can convince agents to write decent behavioral tests if you ask them to with the right structure. Docs are now cheap and a render, not a long term thing. There is some token efficiency to consider, but still, they are quick and cheap if you don't understand some module or its purpose.