21–30 of 120 posts
Snake oil. Just ask the model, all these custom agents/skills haven't proven that useful in practice.
The waterfall approach is better after trying out TDD especially when you have a multi-agent setup. Also I found that in some cases the tests were just superficial hallucinations that never actually tested the components written or there some some context corruption and ultimately triggered a false positive that kicked off a completely unintentional refactoring.
I believe using a skill here is the wrong approach. LLMs already know what TDD is and how to do it, just like object oriented programming. If this is encoded in a skill, that skill essentially has to be loaded for everything thing your LLM is doing. This is probably one of the few areas where direct instructions via AGENTS.md is best, and I don't believe it requires much direction here to force the issue. But I think…
But everybody is free to choose how they work and it may be required in ways that we can't know about.
Test driven development is one of the worst ideas nowadays in the LLM age. We have models that can consistently write expert level, usually bug free code for you and rapidly fix even complex bugs in your codebase. The token cost and tech debt introduced by tests is just not worth it. There's usually no bugs and if there are, you can fix them quickly if and when it's needed.
Testing was and is still very important, as LLMs can still miss important points in business logic or other edge cases I would argue that tests became as important as code, if not more.
TDD sounds great on paper for agentic development but you quickly realize it balloons the token cost. Often I write some feature and then its repurposed or removed, code is refactored moved around as time goes. With TDD I would be taxed heavily and velocity slow to a crawl. The waterfall approach is better after trying out TDD especially when you have a multi-agent setup. Also I found that in some cases the tests wer…
I have to push back on the idea that token costs balloon when using TDD within the context of a strong framework such as Jason has laid out here.
If the feature is repurposed/removed/refactored....I'd argue the specification wasn't well thought out prior to burning into tokens.
We're so eager to do a lot of the wrong things quickly, when it may serve us better to do a more precise thing slowly.
Even more so when coding with agents. I think it is the probably the biggest lever to keep AI in guardrails.
(It's also why I wrote my latest book, Effective Testing, because I routinely find that my clients are very poor at treating.)
TDD sounds great on paper for agentic development but you quickly realize it balloons the token cost. Often I write some feature and then its repurposed or removed, code is refactored moved around as time goes. With TDD I would be taxed heavily and velocity slow to a crawl. The waterfall approach is better after trying out TDD especially when you have a multi-agent setup. Also I found that in some cases the tests wer…
Crazy times here in the development world. I'm always curious to watch other's best practices.
This article would benefit from a date. It looks like it's recent (Internet Archive first grabbed it on May 29th) but it's the kind of information that can quickly become stale as models and agents improve. (I've been getting solid results recently from simply telling Claude Code and Codex "Test with uv run pytest, use red/green TDD".)
# Python Tooling
- Use `uv` to manage Python environments and dependencies.
- Use `uv run` to execute Python scripts and commands.
- Use `pytest` for testing your code.
- Use the `hypothesis` library for property-based testing when you have complex input spaces or need to test edge cases.
- Don't edit `pyproject.toml` directly. Instead, use `uv add` and `uv add --dev` to manage dependencies.
- Use ruff, ty, prek, wily for code quality and linting.
- Don't use excessive casting. If you find yourself needing to cast types frequently, consider refactoring your code to use more appropriate types. Casting should only be done in boundary layers where you are interfacing with external systems.
- Run appropriate tooling after making changes to your code to ensure it meets quality standards.
- When you come across a bug or regression, think hard about writing a test and also how to create code that will prevent this from happening again in the future.
- When creating a command line interface, add `--verbose` flag that provides logging output useful for debugging issues.
- Before creating code, brainstorm 5 different approaches to solve the problem and sort them by their probable effectiveness. Then, choose the best approach and implement it.
- Use Test Driven Development (TDD) for all code you write. Write tests before writing the implementation code.
- Collect pytest fixtures in a `conftest.py` file to avoid duplication
- Prefer testing real code where possible. Use doubles and `monkeypatch` when absolute necessary. Try to avoid mocking as much as possible.
- Favor pytest monkeypatch to mock.
- When a test fails, run the last failed test first using `uv run pytest --last-failed`
- Use numpy-style docstrings for all functions and classes you create.
- Include doctests in the docstrings of your functions to provide examples
- Use type hints for all function parameters and return types.
- Use logging to provide insight into failures. Don't use print for debugging. Don't use logging to hide stack traces.