One trick I've found that works well is to tell it to refactor, e.g for Python: Refactor the Python code to make it more Pythonic, e.g. fewer classes/singletons, especially if it will provide a speedup. The Python code **MUST** follow code organization standards expected of popular open-source Python packages code without causing any benchmark performance regressions. A variant I've used for Rust code: The Rust codeb…
Asking it to apply the YAGNI principle also sems to work well for trimming codebases down. Generally ask it to review, generate a list of review points, then we go through each one together and I make a decision yes/no on each one (or suggest further modifications).
Even if agents can learn to navigate all the stubs and WET crap they leave behind do we really want a code base that no human can follow what's actually going on?
If anything I've seen them go too far with DRY. Like two small functions have logic that you could separate into a shared helper, but no human programmer would do that because it's an unclean abstraction and breaks next time you want to alter either one at all.
One trick I've found that works well is to tell it to refactor, e.g for Python: Refactor the Python code to make it more Pythonic, e.g. fewer classes/singletons, especially if it will provide a speedup. The Python code **MUST** follow code organization standards expected of popular open-source Python packages code without causing any benchmark performance regressions. A variant I've used for Rust code: The Rust codeb…
the word your looking for is idiomatic
The word you are looking for is "you're".
(Can we not play language police? It's boring and doesn't lead to interesting discussion.)
Agents struggle with this: DRY means they create helper functions and classes for a testing scenario and maybe it's called two times. When you ask for a refactor of your code and tests to follow, these helper functions are also ignored and then, dead code starts piling up. Specific cleanup sessions always seem to leave residues behind and doing Ralph loops in the first place seems to help with this, but I'm just not satisfied with the overall performance. Any ideas?
One trick I've found that works well is to tell it to refactor, e.g for Python: Refactor the Python code to make it more Pythonic, e.g. fewer classes/singletons, especially if it will provide a speedup. The Python code **MUST** follow code organization standards expected of popular open-source Python packages code without causing any benchmark performance regressions. A variant I've used for Rust code: The Rust codeb…
Have you tried telling it:
“Write perfect code, make no mistakes”
I use this one in my Ralph Harness all the time, it’s a classic!
It’s not that it can’t do that, it’s just that you haven’t told it to!
Agents struggle with this: DRY means they create helper functions and classes for a testing scenario and maybe it's called two times. When you ask for a refactor of your code and tests to follow, these helper functions are also ignored and then, dead code starts piling up. Specific cleanup sessions always seem to leave residues behind and doing Ralph loops in the first place seems to help with this, but I'm just not…
Use code quality tools. For TypeScript projects I use Fallow gated with commit hooks. Fallow supports checking dead code, complexity hotspots, duplicate lines etc. Works pretty well
If the models are quantifiably different on cleaner codebases, surely this could be leveraged into a measure of code cleanliness.
I'm not sure if you would call it an objective or subjective measure, a fixed model would be consistent which would provide an objective base for comparison, but other models would be different, so it would be subjective in relation to the model itself.
Interesting question to study, but I'm extremely skeptical of the experimental design. They used Opus 4.6 to synthetically produce "degraded" or "cleaned" code bases for relative comparison in the experiment.
Worse, they don't control for breaking the application's tests.
> Pass rate scores the agent’s final state against the hidden tests we wrote for each task. We do not check whether the agent broke unrelated tests already present in the repository, and a cleaner-side and messier-side solution that both pass the hidden test may still differ on tests they were not graded on.
Any conclusions with respect to token consumption seems pretty meaningless if we're not controlling for the quality of the final output.
In my experience, the delta in agent performance is substantial if the codebase is littered with dead code, redundant code, unreachable fallbacks, leaking abstractions and half-baked design patterns vs if the code is well-organized, with clear data flow, with good encapsulation and clean architecture. Like, I've seen all the frontier models have to do several rounds of code review / QA and fix when the code is bad vs…
Feel the same way myself when working in messy codebases… At some point, the horrible patterns start to rub off…
Every time this subject comes up, there are a bunch of takes along the lines of "would you work on a codebase maintained by agents? they'll mess up the code". And I'm asking myself where these people work, because in 20+ years I've yet to see that pristine state of a project that keeps being pristine after the honeymoon greenfield phase, and 50+ people start working on it. Every project devolves in time, old stuff gets patched in a hurry, someone tries to make it better, learns why certain things were done a certain way, hits some undocumented client needs handled by some arcane combination of code + external systems, and so on. If anything, keeping track of what does what in a project is a task where agents can shine, if only in "ask" mode so you can figure out things quicker. Not to mention onboarding and stuff for new team members.