Earlier quoted context omitted.
There is no free lunch. The amount of prompt writing to give the LLM enough context about your codebase etc is comparable to writing the tests yourself. Code assistance tools might speed up your workflow by maybe 50% or even 100%, but it's not the geometric scaling that is commonly touted as the benefits of autonomous agentic AI. And this is not a model capability issue that goes away with newer generations. But it's…
I don't know if this is true. For example, you can spend a few hours writing a really good set of initial tests that cover 10% of your codebase, and another few hours with an AGENTS.md that gives the LLM enough context about the rest of the codebase. But after that, there's a free* lunch because the agent can write all the other tests for you using that initial set and the context. This also works with "here's how I…
Show HN: Continuous Claude – run Claude Code in a loop
31–40 of 66 posts
Re: Show HN: Continuous Claude – run Claude Code in a loop
#32[flagged]
Especially not for snark purposes - https://news.ycombinator.com/newsguidelines.html.
Re: Show HN: Continuous Claude – run Claude Code in a loop
#33> codebase with hundreds of thousands of lines of code and go from 0% to 80%+ coverage in the next few weeks I had a coworker do this with windsurf + manual driving awhile back and it was an absolute mess. Awful tests that were unmaintainable and next to useless (too much mocking, testing that the code “works the way it was written”, etc.). Writing a useful test suite is one of the most important parts of a codebase…
I find coding agents can produce very high quality tests if and only if you give them detailed guidance and good starting examples. Ask a coding agent to build tests for a project that has none and you're likely to get all sorts of messy mocks and tests that exercise internals when really you want them to exercise the top level public API of the project. Give them just a few starting examples that demonstrate how to…
Re: Show HN: Continuous Claude – run Claude Code in a loop
#34Earlier quoted context omitted.
I find coding agents can produce very high quality tests if and only if you give them detailed guidance and good starting examples. Ask a coding agent to build tests for a project that has none and you're likely to get all sorts of messy mocks and tests that exercise internals when really you want them to exercise the top level public API of the project. Give them just a few starting examples that demonstrate how to…
> I find coding agents can produce very high quality tests if and only if you give them detailed guidance and good starting examples. I find this to be true for all AI coding, period. When I have the problem fully solved in my head, and I write the instructions to explicitly and fully describe my solution, the code that is generated works remarkably well. If I am not sure how it should work and give more vague instru…
Re: Show HN: Continuous Claude – run Claude Code in a loop
#35> codebase with hundreds of thousands of lines of code and go from 0% to 80%+ coverage in the next few weeks I had a coworker do this with windsurf + manual driving awhile back and it was an absolute mess. Awful tests that were unmaintainable and next to useless (too much mocking, testing that the code “works the way it was written”, etc.). Writing a useful test suite is one of the most important parts of a codebase…
An agent does a good job fixing it's own bad ideas when it can run tests, but the biggest blocker I've been having is the agent writing bad tests and getting stuck or claiming success by lobotomizing a test. I got pretty far with myself being the test critic and that being mostly the only input the agent got after the initial prompt. I'm just betting it could be done with a second agent.
Re: Show HN: Continuous Claude – run Claude Code in a loop
#36> codebase with hundreds of thousands of lines of code and go from 0% to 80%+ coverage in the next few weeks I had a coworker do this with windsurf + manual driving awhile back and it was an absolute mess. Awful tests that were unmaintainable and next to useless (too much mocking, testing that the code “works the way it was written”, etc.). Writing a useful test suite is one of the most important parts of a codebase…
There is no free lunch. The amount of prompt writing to give the LLM enough context about your codebase etc is comparable to writing the tests yourself. Code assistance tools might speed up your workflow by maybe 50% or even 100%, but it's not the geometric scaling that is commonly touted as the benefits of autonomous agentic AI. And this is not a model capability issue that goes away with newer generations. But it's…
I recently had a bunch of Claude credits so got it to write a language implementation for me. It probably took 4 hours of my time, but judging by other implementations online I'd say the average implementation time is hundreds of hours.
The fact that the model knew the language and there are existing tests I could use is a radical difference.
Re: Show HN: Continuous Claude – run Claude Code in a loop
#37Earlier quoted context omitted.
I find coding agents can produce very high quality tests if and only if you give them detailed guidance and good starting examples. Ask a coding agent to build tests for a project that has none and you're likely to get all sorts of messy mocks and tests that exercise internals when really you want them to exercise the top level public API of the project. Give them just a few starting examples that demonstrate how to…
Has anyone had success with specific prompts to avoid the agent over-indexing on implementation details? For instance, something like: "Before each test case, add a comment justifying the business case for every assumption made here, without regards to implementation details. If this cannot be made succinct, or if there is ambiguity in the business case, the test case should not be generated."
Starting point: small-ish codebase, no tests at all:
> I'd like to add a test suite to this project. It should follow language best practices. It should use standard tooling as much as possible. It should focus on testing real code, not on mocking/stubbing, though mocking/stubbing is ok for things like third party services and parts of the code base that can't reasonably run in a test environment. What are some design options we could do? Don't write any code yet, present me the best of the options and let me guide you.
> Ok, I like option number two. Put the basic framework in place and write a couple of dummy tests.
> Great, let's go ahead and write some real tests for module X.
and etc. For a project with an existing and mature test suite, it's much easier: > I'd like to add a test (or improve a test) for module X. Use the existing helpers and if you find yourself needing new helpers, ask me about the approach before implementing
I've also found it helpful to put things in AGENTS.md or CLAUDE.md about tests and my preferences, such as: - Tests should not rely on sleep to avoid timing issues. If there is a timing issue, present me with options and let me guide you
- Tests should not follow an extreme DRY pattern, favor human readability over absolute DRYness
- Tests should focus on testing real code, not on mocking/stubbing, though mocking/stubbing is ok for things like third party services and parts of the code base that can't reasonably run in a test environment.
- Tests should not make assumptions about the current running state of the environment, nor should they do anything that isn't cleaned up before completing the test to avoid polluting future tests
I do want to stress that every project and framework is different and has different needs. As you discover the AI doing something you don't like, add it to the prompts or the AGENTS.md/CLAUDE.md. Eventually it will get pretty decent, though never blindly trust it because a butterfly flapping it's wings in Canada sometimes causes it to do unexpected things.Re: Show HN: Continuous Claude – run Claude Code in a loop
#38Re: Show HN: Continuous Claude – run Claude Code in a loop
#39> codebase with hundreds of thousands of lines of code and go from 0% to 80%+ coverage in the next few weeks I had a coworker do this with windsurf + manual driving awhile back and it was an absolute mess. Awful tests that were unmaintainable and next to useless (too much mocking, testing that the code “works the way it was written”, etc.). Writing a useful test suite is one of the most important parts of a codebase…
I find coding agents can produce very high quality tests if and only if you give them detailed guidance and good starting examples. Ask a coding agent to build tests for a project that has none and you're likely to get all sorts of messy mocks and tests that exercise internals when really you want them to exercise the top level public API of the project. Give them just a few starting examples that demonstrate how to…