Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Agent Skill for Test-Driven Development

saturnci.com

Post reply on HN
Print view View original

81–90 of 120 posts

Re: My Agent Skill for Test-Driven Development

#81

Post by cbcjcyv5 »

bob1029 wrote: post #78

TDD is fundamentally problematic in every practical implementation I've ever seen. I don't think the same thing, but much faster, is going to help at all. TDD tends to cause adverse, higher order effects. I am currently observing AI authored tests creating a massive sense of complacency because a human no longer owns responsibility for the test suite. It's too easy to reject ownership by way of the various agent prom…

Tests AI writes aren't for me. They're for the AI.

I mostly agree though, I've seen a lot of vapid assertions in my day job recently.

I should note Im specifically not doing tdd with AI.

Re: My Agent Skill for Test-Driven Development

#83

Post by mpweiher »

emigre wrote: post #79
bfeynman wrote: post #59

Earlier quoted context omitted.

In what world or frame of reference would doing TDD have "little" bearing on output quality? If you build a system around satisfying some set of requirements it seems logical that output quality would have pretty heavy correlation.

It's possible to satisfy a set of requirements with code that's low quality. There's the maintainability of the code, for example, or the performance of the system.

The set of requirements TDD encourages code to meet happen to be ones that increase code quality.

Code that is easy to test tends to be well-structured.

Code that is badly structured tends to be hard to test.

TDD is not a QA methodology, it is a design methodology. It also tends to help quality out a lot, but that's a secondary effect.

Re: My Agent Skill for Test-Driven Development

#86

Post by nextaccountic »

simonw wrote: post #8

This article would benefit from a date. It looks like it's recent (Internet Archive first grabbed it on May 29th) but it's the kind of information that can quickly become stale as models and agents improve. (I've been getting solid results recently from simply telling Claude Code and Codex "Test with uv run pytest, use red/green TDD".)

https://github.com/jasonswett/llm-skills/blob/main/tdd/SKILL... has a timestamp (mar 14, 2026 as of today)

Re: My Agent Skill for Test-Driven Development

#87

Post by oefrha »

porphyra wrote: post #20
simonw wrote: post #8

This article would benefit from a date. It looks like it's recent (Internet Archive first grabbed it on May 29th) but it's the kind of information that can quickly become stale as models and agents improve. (I've been getting solid results recently from simply telling Claude Code and Codex "Test with uv run pytest, use red/green TDD".)

A lot of prompt engineering goes out of date quickly. Nobody nowadays goes "you are an expert software engineer. make no mistakes" lol. As a personal anecdote, I find that a lot of big prompts and skills use up context window budget and in many cases agents will eagerly try to use a skill even if it isn't super relevant or necessary for the current task. So when I have too many skills I have to spend a bunch of time…

> Nobody nowadays goes "you are an expert software engineer. make no mistakes"

You know what, I checked Opus 4.8's instructions to a review subagent the other day and it literally opened with

> You are a senior infrastructure/security engineer doing a thorough, adversarial code review...

I didn't say anything like that myself.

Re: My Agent Skill for Test-Driven Development

#88

Post by esperent »

rsalus wrote: post #42
zuzululu wrote: post #23

TDD sounds great on paper for agentic development but you quickly realize it balloons the token cost. Often I write some feature and then its repurposed or removed, code is refactored moved around as time goes. With TDD I would be taxed heavily and velocity slow to a crawl. The waterfall approach is better after trying out TDD especially when you have a multi-agent setup. Also I found that in some cases the tests wer…

I was a big proponent of encoding TDD red-green-refactor methodology into my agent workflows until recently when I made the same realization after reading this study: https://arxiv.org/pdf/2602.07900 TLDR; it found test-writing volume only weakly correlates with success and that encoding test-writing principles did not move resolution rates but _did_ materially change cost. Encouraging tests cost +19.8% output tokens…

From that paper:

> This raises a central question: do such tests meaningfully improve issue resolution, or do they mainly mimic a familiar software-development practice while consuming interaction budget?

This is an important question but it's not the one I'm most interested in when requiring agents to follow TDD. My goal is to lock in behavior because it was happening way too frequently that an agent would successfully fix the issue at hand, but break something else that it wasn't supposed to touch.

The tests add another layer and it's why I always separate out red and green worker subagents. The green worker might get trigger happy and go beyond scope/break something but it's not allowed to fudge the tests so I'll know and can clean up and revert.

It's also why I'm not too bothered about perfect red green TDD. I can add the tests later if needed.

Re: My Agent Skill for Test-Driven Development

#89

Post by krupan »

dominotw wrote: post #18
calebkaiser wrote: post #16

Earlier quoted context omitted.

I don't understand this line of criticism exactly. By putting new information in the context window, you are materially changing the activations at your point of sampling, which is literally "customizing with mere markdown files." Taken to the extreme, the attitude that there is some special incantation that will unlock all capabilities is silly, and a lot of the "prompt engineering" discourse is similarly kind of du…

even if that works one time you can never be sure that your customization is in place or fell out of context's important zone or is contradicted by later context . you've reverted back to base llm behavior. you are treating skill like sure thing

It's as of people crave some sort of control and/or determinism from these chaotic tools, and so they have to believe that these "skills" make a difference

Re: My Agent Skill for Test-Driven Development

#90

Post by krupan »

john_strinlai wrote: post #4
behnamoh wrote: post #2

Snake oil. Just ask the model, all these custom agents/skills haven't proven that useful in practice.

not that i know much about the effectiveness of these skill files, i find it odd to call something given for free "snake oil", which i thought referred to the sale of fraudulent products (to the benefit of the snake oil salesperson), typically around healthcare-related stuff.

If the consume more tokens then they are not free. If they consume those tokens without really improving anything them they are snake oil
Post reply on HN