This is a decent approach. My concern with TDD is that writing tests necessarily implies designing an API upon which those tests operate. Here, the agent is instructed to "not write code, write tests", and yet, in doing so it defines an API. This will cause the AI to hallucinate the API. Layering in yet more tests on top of this will cause that API to deform in strange ways that pass tests but that the adversary will…
> My concern with TDD is that writing tests necessarily implies designing an API upon which those tests operate It really forces you to do outside-in testing; I usually describe the kind of API I want when chatting with the agents. For example, the CLI options, the routes that might be useful for an API, etc. > I look at the code and find a rats nest / ball of mud that will cost 10x more tokens to enhance should I ev…
- write a test for method that does not exist, it just calls the method and nothing else
- write method that does nothing
- add/extend test that uses that method - modify method until tests passes
- go back to loop start until you're done
I always hated it. When I work with LLM i first massage interface that tests, then tests, then implementation until all these tests pass.
> for example when it re-outputs the complete file when you ask for a tiny change).
well with sonnet 3.5 and 4.5 (can't say about 4.6) it often will get stuck in a loop trying to update just the required parts and iether waste tons of tokens doing these updates or waste tons of tokens to a point where restring file from git is required. Tokens get wasted regardless.