Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How well do agents use test/verification techniques?

danluu.com

Post reply on HN
Print view View original

51–60 of 83 posts

Re: How well do agents use test/verification techniques?

#51

Post by andai »

I'm doing indie game dev, so not sure how much my experience transfers, but I've been working on a browser game and the game kind of sucks, but there's a huge amount of tests (by my standards). So you can have crap software with loads of tests. (You can also have awesome software with very few tests!)

Also, had a funny experience where AI implemented an architectural change completely backwards. The implementation was pointless and made things worse rather than better. But I still got "all tests green" lol, because it just proved that the incorrect thing worked properly.

I noted with some amusement that formal verification wouldn't have helped there either, it would just have been an even stronger proof of the "correctness" of the thing that shouldn't exist to begin with.

Re: How well do agents use test/verification techniques?

#52

Post by pydry »

jiaosdjf wrote: post #31
ivanzhaowy123 wrote: post #18

In my experience, agents often think of more edge cases than humans when writing unit tests. But under the guidance of certain skills, they can become mechanical and lose sight of the business logic. For example, when I use the Superpowers skill set, the agent proactively adopts TDD for every new feature. But its understanding of testing often stays superficial: if the user asks for a screen with a “Send” button, it…

Probably because thats how most companies do it because most corporate workers are lazy box-tickers who are long fed up with the processes. It really feels like we're forcing human processes onto AI. What actually is the point of TDD? - If its to force you to think about edge cases early before you've started building the feature then that sounds like a human trait - If its to be living documentation then that sounds…

I think most people miss the point of TDD. It is not just about edge cases. It ought to be:

* A way of matching requirement use cases to tests.

* to modulate the number of tests written not only to make sure you have enough coverage but also to make sure you don't pointlessly cover the same edge cases multiple times.

* a way to cheaply provide feedback and validation on code as you are writing it.

With AI the ability to churn out useless tests has exploded (both with TDD done badly and with no TDD at all) and that has actually incurred a new type of cost we didnt have to face before.

Re: How well do agents use test/verification techniques?

#53

Post by andai »

ivanzhaowy123 wrote: post #18

In my experience, agents often think of more edge cases than humans when writing unit tests. But under the guidance of certain skills, they can become mechanical and lose sight of the business logic. For example, when I use the Superpowers skill set, the agent proactively adopts TDD for every new feature. But its understanding of testing often stays superficial: if the user asks for a screen with a “Send” button, it…

I asked Sonnet to port a prolog interpreter from Python to JS. I was surprised and delighted that it ended up writing a test suite for it. All tests passed.

Until... I inspected the code: it was just a bunch of print statements that said "test passed!" and didn't actually test anything.

That was last year, so hopefully it doesn't do that anymore.

Re: How well do agents use test/verification techniques?

#54

Post by yorwba »

kqr wrote: post #25
vetronauta wrote: post #4

If I understood correctly, the author had the agent perform manual mutation testing (write code, write passing tests, manually change the code to see some tests fail, then revert the change), rather than automated mutation testing. Why not use a mutation-testing framework and consider the build failed if a certain percentage of mutations are not killed? The article claims that the agents didn’t actually use TDD or mu…

This surprised me too. I suspect the robots only got a "use mutation testing" prompt and independently decided that it must mean manual mutation testing, or possibly that they ran in a sandbox where an automated mutation testing tool was not installed. But I'm surprised Dan Luu didn't make a remark about this. Surely he must know the difference!

He did remark on it: "Although mutation testing is a standard programming term, agents generally didn't actually do mutation testing and instead did normal testing with some small amount of mutating things in a way that isn't really mutation testing, similar to how the TDD instruction modified behavior but didn't get agents to do TDD."

If you're wondering why he doesn't go into greater detail, several testing methodologies earlier we have: "Since just saying that agents didn't really meaningfully do the thing is repetitive, I'll make these sections short and only highlight particular curiosities."

Re: How well do agents use test/verification techniques?

#55

Post by nseskin »

I think this article demonstrates a broader problem with constraint following in LLMs. Agents often satisfy the most obvious and easily verifiable part of a task, but lose track of the constraint that actually determines success.

Testing makes this especially clear: “use fuzzing” turns into generating random bytes, and “use formal verification” into proving a property that isn’t particularly useful. Formally, the technique is applied, but its actual purpose is lost.

This may also explain why different testing techniques produce similar results: the problem may be less about knowing a particular technique and more about the agent’s ability to keep its actual goal as the primary constraint.

Re: How well do agents use test/verification techniques?

#56

Post by CharlieDigital »

I worked for a long stint building SaaS for life sciences/pharma.

A big, big part of this space is "traceability" through the SDLC. If something goes wrong, there is a long chain of liability from the final effect (a subject with an adverse event) to the root cause (system recorded the wrong data in a clinical trial) all the ways to a 3rd party vendor. We call software "validated" if it has gone through the GAMP 5 V-model of software development and produced the necessary artifacts that correlate verified behavior to specification.

While this is way too much rigor for many scenarios, I think more folks should be familiar with the GAMP 5 V-model in the agentic era. The left side of the V defines the requirements moving from high level business requirements to low level technical requirements, the right side of the V defines the verification artifacts corresponding to the technical and business requirements (in that order). In this structure, testing covers both the functional requirements as well as the technical requirements.

I think this mental model is extremely useful and reflects the interaction model with agents: humans now focus on the requirements and less on the implementation details. It is useful to separate the business from the technical and therefore, the two types of artifacts that need to be produced to satisfy verification thresholds.

This style of structured software development (pre-agents) is very expensive. With a 2-3 weeks planning/specification phase on the front-end (since the test specification has a dependency on the approved requirements), a 4 week implementation phase (this is iterative with the test team, but informal; changes from the approved plan are documented as deviations), and then a 2-3 week formal test/verification phase. But in a post-agent world, this model feels like it is 1) more reasonable, 2) perhaps more effective, 3) more manageable.

While the teams I've now been on in startups have focused heavily on fast iteration with AI, a big downside I've noted is that it seems that we keep building the wrong things or things that are not useful because we are no longer verifying the business requirements before building. We assume the cost of building is low so we build and the iterate; testing in this reality becomes ad-hoc and full of gaps because the functional behavior of the software is no longer scoped before building; it's "vibes" based on each iteration because the iterations are cheap. The result? The teams I've been on feel like we're going nowhere in many cases, just spinning faster and producing more throwaway code (not a comment on whether this is good or bad; it may vary by domain and nature of the company).

If one is looking to build a software factory or pipeline, I think it behooves an architect to examine the GAMP 5 V-model and consider how to adapt ideas from this model to agent systems.

Re: How well do agents use test/verification techniques?

#57

Post by gregwebs »

Amazing work! But deeply frustrating on the lack of reproducibility and how far off this is from a proper SDLC.

How can I inspect exactly how agents were prompted? How can I reproduce this setup and try out my own AI setup? Am I missing a link to a repo somewhere?

These evals probably match how many people are using AI, but its not the full package of how we know software development needs to be done, and not how I do it with AI. The closest would be "Audit" which scored the highest when using xhigh- that actually incorporates a review cycle- something we know is the most important part of the software development process for code correctness (design/specification is not as much of an issue in this problem since the task is to write code against an existing spec). However, we don't know what instructions they have in their "Audit".

I would love to benchmark my own flow [1] if I can be given their exact problem. What it does is (assuming there is already a solid spec)

  * plan with expensive model. Review the plan.
  * implement with cheap model. Review for spec compliance and code quality.
  * Reviews are done adversarially from the expensive model with a fresh context.
  * ensure that verifications (automated or manual) are performed.
For non-trivial changes, the review and verification process almost always catch significant issues.

The workflow does use TDD. I do find useless tests being written and I need to dig into this part of the workflow a lot more, so its great to see that aspect of this article. My experience writing software has taught me that code must be written to be easy to test, but not necessarily done TDD style.

[1] https://github.com/gregwebs/skills-sdlc/

Re: How well do agents use test/verification techniques?

#58

Post by grohan »

nseskin wrote: post #55

I think this article demonstrates a broader problem with constraint following in LLMs. Agents often satisfy the most obvious and easily verifiable part of a task, but lose track of the constraint that actually determines success. Testing makes this especially clear: “use fuzzing” turns into generating random bytes, and “use formal verification” into proving a property that isn’t particularly useful. Formally, the tec…

Pretty good point - I would actually argue that the imperative nature of prompting takes some of the blame here too. Your initial goal gets "steered" by the agent system and follow-up messages take precedence.

Re: How well do agents use test/verification techniques?

#59

Post by williamdclt »

ckvibubueu wrote: post #27
ngruhn wrote: post #12

I wonder how well this fares try to make illegal states unrepresentable In my experience, agents know how to do it. They just don't if it's not the default style of the language.

Go is an amazing AI language for this reason. The tests that get created also tend to be higher quality than say the slop I see in python. Though I do suspect my codebases are doing heavy lifting in terms of steering towards quality outcomes.

> Go is an amazing AI language for this reason.

I'd be interested to hear more about why? because my experience with Go has been the opposite, I found it pretty bad for "making illegal states unrepresentable". In fact, its zero-values system often makes illegal states the _default_

Re: How well do agents use test/verification techniques?

#60

Post by Schlagbohrer »

andai wrote: post #51

I'm doing indie game dev, so not sure how much my experience transfers, but I've been working on a browser game and the game kind of sucks, but there's a huge amount of tests (by my standards). So you can have crap software with loads of tests. (You can also have awesome software with very few tests!) Also, had a funny experience where AI implemented an architectural change completely backwards. The implementation wa…

I run in to this often with my Qwen3.8-27B agent. I roll my eyes when it reassures me "467/467 tests green, it's good to go!" and there is some basic misunderstanding or the output looks like crap.
Post reply on HN