51–60 of 112 posts
Earlier quoted context omitted.
the word your looking for is idiomatic
The word you are looking for is "you're". (Can we not play language police? It's boring and doesn't lead to interesting discussion.)
Earlier quoted context omitted.
> Some of the issues mentioned above like dead code removal, code duplication, unreachable code are already solved using deterministic linters for quite a while now for most language ecosystems. I have legacy endpoints that are no longer used in practice, there for historical reasons, intertwined with existing code etc. They might be marked obsolete, services implementing it are not - agent greps those, builds off of…
Yeah, we have a big struggle with this. We have lots of legacy code that doesn't follow our latest design patterns intermixed with new code. The LLM picks up bad habits depending on what it pulls in to context first. We have AGENTS.md configured with the right way, but old style still slips in. We obviously need to update the old code but on the other hand if it ain't broke why touch it.
// LEGACY CODE, per docs/legacy_rules.md §14, §19In my experience, the delta in agent performance is substantial if the codebase is littered with dead code, redundant code, unreachable fallbacks, leaking abstractions and half-baked design patterns vs if the code is well-organized, with clear data flow, with good encapsulation and clean architecture. Like, I've seen all the frontier models have to do several rounds of code review / QA and fix when the code is bad vs…
Some of the issues mentioned above like dead code removal, code duplication, unreachable code are already solved using deterministic linters for quite a while now for most language ecosystems. You can get the LLM to run a script which checks for all of these and also enforce them by running the same script as a pre-commit hook. Setting this up religiously in every code base I work on has been what's given me the most…
Most of the time. Except for when it forgets to do it.
I think it’s funny that the solution is to use something that is not LLM driven to enforce it.
Also - pre commit hooks aren’t enforced, people will not set them up. You have to run this stuff in CI (which is incredibly annoying given that machines are writing the code in the first place)
Earlier quoted context omitted.
I’ve been working with these things for quite some time now and every time I simply “treat it like I would a human” it seems to perform better. I can’t imagine agents wouldn’t perform better in a clean codebase than a giant mess of one. Just like it performs better when it has well formed specs and access to documentation.
It actually goes even further than humans, humans can pretty rapidly get inured to things being awkward or messy and stop noticing, but the context for agents is taking up the same space and "attention" every time they're run, and they're creations entirely of context, so the quality and examples matter massively.
You’ve never had an agent completely lose the plot and forget/confuse its instructions due to the context filling up?
Earlier quoted context omitted.
Some of the issues mentioned above like dead code removal, code duplication, unreachable code are already solved using deterministic linters for quite a while now for most language ecosystems. You can get the LLM to run a script which checks for all of these and also enforce them by running the same script as a pre-commit hook. Setting this up religiously in every code base I work on has been what's given me the most…
> You can get the LLM to run a script which checks for all of these Most of the time. Except for when it forgets to do it. I think it’s funny that the solution is to use something that is not LLM driven to enforce it. Also - pre commit hooks aren’t enforced, people will not set them up. You have to run this stuff in CI (which is incredibly annoying given that machines are writing the code in the first place)
"agent pipelines that [...] clean a messy [repository]" This feels like a terrible approach, sufficient to condemn the entire study. Apparently half of the "minimal pairs" in this work were constructed in this way. I simply am not going to trust any conclusion that requires assuming these AI "cleaned" repos are in any way representative of actually-good codebases.
Using LLMs to rewrite code to remove these violations is a rather accepted practice. Sonar's existing one-shot LLM based approach [1] (in production since 1+ year), and a recent agentic approach [2] to do the same work rather well to do this.
[1] https://www.sonarsource.com/solutions/ai/ai-codefix/
[2] https://www.sonarsource.com/products/sonarqube/remediation-a...
Earlier quoted context omitted.
> You can get the LLM to run a script which checks for all of these Most of the time. Except for when it forgets to do it. I think it’s funny that the solution is to use something that is not LLM driven to enforce it. Also - pre commit hooks aren’t enforced, people will not set them up. You have to run this stuff in CI (which is incredibly annoying given that machines are writing the code in the first place)
This is the most frustrating part. You do everything you can to ensure there are clear instructions, you can keep the agent MD as concise and clear and short as possible. It still feels like it's all just a suggestion, and of course it is, because it's all just another part of the prompt.