Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Agent Skill for Test-Driven Development

saturnci.com

Post reply on HN
Print view View original

31–40 of 120 posts

Re: My Agent Skill for Test-Driven Development

#31

Post by dools »

__mharrison__ wrote: post #29
zuzululu wrote: post #23

TDD sounds great on paper for agentic development but you quickly realize it balloons the token cost. Often I write some feature and then its repurposed or removed, code is refactored moved around as time goes. With TDD I would be taxed heavily and velocity slow to a crawl. The waterfall approach is better after trying out TDD especially when you have a multi-agent setup. Also I found that in some cases the tests wer…

My experience is the opposite. TDD keeps the guardrails on and let's me refactor with confidence. Crazy times here in the development world. I'm always curious to watch other's best practices.

Yeah I specifically tell it not to pre-emptively fix tests that it knows will break as a result of changes its making and instead limit itself only to creating new tests for new changes. I want to see the tests break, then we go through and review each set of breakages versus the mission and assess if they’re regressions or stale assertions. This is a) how I know it’s actually writing meaningful tests b) a very functional and useful form of “code review” versus just trying to catch problems by reading diffs and c) helped me find real problems and regressions.

Almost all the breakages after a big refactor are stale assertions but every time I catch a couple of critical problems that make the entire exercise very worth it.

The whole dev process is so fast compared to writing software manually that I find it absurd that I wouldn’t invest heavily in automated tests.

Re: My Agent Skill for Test-Driven Development

#32

Post by __mharrison__ »

dools wrote: post #31
__mharrison__ wrote: post #29

Earlier quoted context omitted.

My experience is the opposite. TDD keeps the guardrails on and let's me refactor with confidence. Crazy times here in the development world. I'm always curious to watch other's best practices.

Yeah I specifically tell it not to pre-emptively fix tests that it knows will break as a result of changes its making and instead limit itself only to creating new tests for new changes. I want to see the tests break, then we go through and review each set of breakages versus the mission and assess if they’re regressions or stale assertions. This is a) how I know it’s actually writing meaningful tests b) a very funct…

See my AGENTS.md in nearby comment

Re: My Agent Skill for Test-Driven Development

#33

Post by zuzululu »

reg_dunlop wrote: post #27
zuzululu wrote: post #23

TDD sounds great on paper for agentic development but you quickly realize it balloons the token cost. Often I write some feature and then its repurposed or removed, code is refactored moved around as time goes. With TDD I would be taxed heavily and velocity slow to a crawl. The waterfall approach is better after trying out TDD especially when you have a multi-agent setup. Also I found that in some cases the tests wer…

But that repurposing/removal is exactly what's avoided if you follow through with the SEF framework he outlines. I have to push back on the idea that token costs balloon when using TDD within the context of a strong framework such as Jason has laid out here. If the feature is repurposed/removed/refactored....I'd argue the specification wasn't well thought out prior to burning into tokens. We're so eager to do a lot o…

You cant spec out what you dont know, scope, requirements change from real world feedback

Re: My Agent Skill for Test-Driven Development

#34

Post by jzig »

zuzululu wrote: post #23

TDD sounds great on paper for agentic development but you quickly realize it balloons the token cost. Often I write some feature and then its repurposed or removed, code is refactored moved around as time goes. With TDD I would be taxed heavily and velocity slow to a crawl. The waterfall approach is better after trying out TDD especially when you have a multi-agent setup. Also I found that in some cases the tests wer…

Pattern-based testing can theoretically reduce the token cost?

Re: My Agent Skill for Test-Driven Development

#35

Post by manmal »

zuzululu wrote: post #23

TDD sounds great on paper for agentic development but you quickly realize it balloons the token cost. Often I write some feature and then its repurposed or removed, code is refactored moved around as time goes. With TDD I would be taxed heavily and velocity slow to a crawl. The waterfall approach is better after trying out TDD especially when you have a multi-agent setup. Also I found that in some cases the tests wer…

> With TDD I would be taxed heavily and velocity slow to a crawl.

And the code will be good.

Re: My Agent Skill for Test-Driven Development

#36

Post by fowlie »

Haven't tried this, but I've recently become a big fan of Matt Pococks skills. Workflow: /grill-with-docs -> /to-prd -> /to-issue -> /tdd. That will interview relentlessy until there is a "shared understanding" using "ubiquitous language", then it will spec all requirements with user stories, create issues and implement them using tdd.

Re: My Agent Skill for Test-Driven Development

#38

Post by enraged_camel »

Spawning separate agents to review the original agent's implementation results in a very noticeable increase in code quality and decrease in bugs. This is why I encode two or three rounds of sub-agent review during the planning process, where I tell the agent authoring the plan to include those review rounds at the end. If the code is particularly load-bearing, I then ask a fourth agent, usually from the other frontier lab.

All of this burns more tokens of course, but probably way less than coming back to the code later to fix bugs. It is also slower, but in the long run saves time.

Re: My Agent Skill for Test-Driven Development

#39

Post by esafak »

steno132 wrote: post #17

Test driven development is one of the worst ideas nowadays in the LLM age. We have models that can consistently write expert level, usually bug free code for you and rapidly fix even complex bugs in your codebase. The token cost and tech debt introduced by tests is just not worth it. There's usually no bugs and if there are, you can fix them quickly if and when it's needed.

IF your code has no bugs it's either trivial or you haven't noticed the bugs.

Re: My Agent Skill for Test-Driven Development

#40

Post by bisonbear »

jvuygbbkuurx wrote: post #25

All of these post are missing actual comparisons on results. I read exactly opposite 'you should do x' everyday. If TDD actually was better it would simply be in the system prompts already.

Agree - all of this is based on vibes (I also use TDD based on vibes FWIW). The only way to settle "does TDD / caveman / [insert random skill here] help" is to replay real PRs from your repo and measure quality
Post reply on HN