How do you guys ensure that the refactoring improves the existing code?
The answer to "how do you ensure refactoring improves code?" is embedded in the binary as a system prompt. It's his own blog post about the Embedded Design Principle.
The binary contains 9 system prompts, all instruction templates for the LLM. None contain any code for measuring code quality (unfortunately)
The pipeline is three steps:
suggest-data-unifications - prompts the LLM with the blog post. The prompt starts literally with "For each data structure in the specified code, do the following."
suggest-code-unifications - same agent, different prompt. Starts with "Now look at the file and apply the above guidelines."
execute-refactoring - runs the LLM's suggestions through a coding agent.
No verification between steps. No quality gate. No baseline comparison.
The refactoring agent's entire context is the blog post, literally. Read it. Find duplication. Merge it.
The closest thing to a "guardrail" is a function which calls eval() on arbitrary user-defined JavaScript. And AutoAcceptDecorator which intercepts LLM messages matching /proceed|go ahead|make|implement|apply/ and auto-replies "Yes, please proceed with the changes."
So when you ask "how do you ensure it improves code?" the answer is: we ask an LLM to read a blog post about code quality and then we trust it. And we built a regex that auto-accepts its own changes.
The binary also has a separate class for fiber-based refactoring execution, and a full walkthrough generation pipeline that auto-generates code walkthroughs from git diffs. There's a separate workflow for file organization that reads Jimmy Koppel's rule ("Make the design apparent in the code") and applies section headers to changed files. Completely independent from the deduplication agent but uses the same pipeline: read prompt, LLM, apply changes.
And the DoItAll workflow chains everything together. DeDuplicate runs in parallel, then embedded-design and organize-file run on every changed file with concurrency:2. It's a full refactoring pipeline.... but every single step is just: read a blog post, LLM, apply. The entire product is two blog posts, a concurrency manager, and a regex.