[flagged]
How well do agents use test/verification techniques?
61–70 of 83 posts
Re: How well do agents use test/verification techniques?
#62You know how everyone thinks agents are bad at the thing they're good at and good at the thing they're bad at? It turns out I must be bad at testing, because I thought they do a reasonable job.
Isn't that just a form of Dunning-Kruger effect related to the parts that person is bad in? It looks good because they don't have the depth of knowledge/expertise to see the deficiencies.
Re: How well do agents use test/verification techniques?
#63I think this article demonstrates a broader problem with constraint following in LLMs. Agents often satisfy the most obvious and easily verifiable part of a task, but lose track of the constraint that actually determines success. Testing makes this especially clear: “use fuzzing” turns into generating random bytes, and “use formal verification” into proving a property that isn’t particularly useful. Formally, the tec…
It's well known that CoT needs a narrow problem or several independent narrow problems to perform well. With too broad of a task it often turns into a burden and hurts performance, making it hyperfocus on one thing where a non-reasoning model would just handle everything. I wonder if disabling it would make anything better, but likely not if the model is already trained to do it.
Re: How well do agents use test/verification techniques?
#64Amazing work! But deeply frustrating on the lack of reproducibility and how far off this is from a proper SDLC. How can I inspect exactly how agents were prompted? How can I reproduce this setup and try out my own AI setup? Am I missing a link to a repo somewhere? These evals probably match how many people are using AI, but its not the full package of how we know software development needs to be done, and not how I d…
As someone who also spent a bunch of time benchmarking various techniques, this is as good as you can do without publishing a formal versioned benchmark suite that you want to maintain and run forever at immense cost to yourself. If you actually go to benchmark your own flow as you mentioned, you will quickly run into like a dozen problems that discourage you from publishing.
- Are you sure that temperature and other nondeterminism isn't affecting your output?
- Are you sure you're not being routed through an A/B test at this moment?
- Are you sure there's not a bug affecting the model at this moment?
- Are you sure that you picked the right model and effort level?
- Are you sure that your result generalizes across providers?
- Are you sure that you set up the correct level of sandboxing and the agent can't e.g. look at a sister directory or git history in the current directory for answers?
- Are you sure that the agent isn't leaking answers in memory or its conversation history?
- Are you sure that tool calls aren't somehow affecting results?
- Are you sure that your results are robust, i.e. you see the same results with mild tweaks to the prompt?
- Are you comfortable keeping your blog post live when your results are invalidated next week with the next model launch?
And that's just a quick list off the top of my head.
I personally decided that it wasn't worth it, I'm glad that Dan decided to publish his. Frankly I think we could use a lot more of these "I ran these 2 techniques side by side and here's what I saw" anecdata, because most people who promote prompt techniques can't produce a single prompt they ran twice because they never actually tested it per se.
Edited to add: formatting + the word "promote"
Re: How well do agents use test/verification techniques?
#65Audit is the way. Even better is having one model family like Claude check another model family like Codex.
Re: How well do agents use test/verification techniques?
#66It is still early, but I find that this experiment makes little to no sense and it is barely useful. The way you test code cannot (and should not) be decoupled by the way in which you architect the code itself. 80%+ of effective testing is not in the testing framework but in the code architecture. The author doesn't mention how the code is being architected and managed. For what it is worth, I found that forcing agen…
> The way you test code cannot (and should not) be decoupled by the way in which you architect the code itself. I'm not sure I fully understand. In my view, it's pretty evident that tests should test behavior and not structure. Coupling to implementation details is unavoidable but should be managed, and ideally minimized, to keep the test suite robust and maintainable. Given the choice between a test suite that monke…
> I'm not sure I fully understand. In my view, it's pretty evident that tests should test behavior and not structure.
Not OP, but I think I know what OP has in mind. There are some trivial ways in which code can be made easier to test as well as more complex (let's call them architectural) ways.
Examples of trivial conveniences include: making tested functionality accessible from the test (i.e. private class methods in Java present a problem for testing), coding against an interface rather than an implementation (this allows substituting a test implementation for a production implementation that can fish out some bugs), having internal to the code correctness checks that don't necessarily influence program execution (simply having runtime type checks, like it's done in eg. Java, compared to eg. C makes testing easier and more productive). Of course, there's more, these are just very common and easy to understand.
The architectural properties that improve the ability to test a program might be: the so-called "observability" programmed into the product, i.e. the product generates metrics data that's not part of the desired output. The product is split into modules with formally defined interfaces, which allows testing modules in isolation and lowers the number of possible combinations to test.
Improving testability isn't necessarily a good thing because it's likely to negatively impact complexity, size or speed of the program. So, depending on what's more important for any given program, the testing strategy will be different...
Re: How well do agents use test/verification techniques?
#67Results seem somewhat reasonable given that the amount of verus/TLA/Creusot/Lean code out there is tiny compared to all the other non-formal code. So it's understandable that the agents wont be able to go beyond proving trival things, given how much more difficult it is to write such code. A (more) interesting experiment (to me) would be to write a high level spec manually for a non-trivial system (liveness etc.) and…
Picking rust also felt problematic as also somewhat out of distribution historically
Re: How well do agents use test/verification techniques?
#68Earlier quoted context omitted.
But what's the initial setup? Was it greenfields every time? And _how_ were the different testing frameworks used? What's the AGENTS.md there? So many things can influence these tests in positive/negative ways that are not included in the write up or results. Seems like a lot of effort without much in the way of actual helpful detail.
The setup is described in the linked post about programming language vs efficiency/token cost: https://danluu.com/pl-tokens/#zstd > For the first eval, I tried giving agents the zstd RFC (plus errata) and telling them to implement a complete zstd decoder (agents are stuck in a container without internet access). The tests were not given to agents. For something with the surface area of zstd, it's not really reasonabl…
agents are stuck in a container without internet access. The tests were not given to agents
It's a bold strategy, cotton.
Re: How well do agents use test/verification techniques?
#69I think this article demonstrates a broader problem with constraint following in LLMs. Agents often satisfy the most obvious and easily verifiable part of a task, but lose track of the constraint that actually determines success. Testing makes this especially clear: “use fuzzing” turns into generating random bytes, and “use formal verification” into proving a property that isn’t particularly useful. Formally, the tec…
It's well known that CoT needs a narrow problem or several independent narrow problems to perform well. With too broad of a task it often turns into a burden and hurts performance, making it hyperfocus on one thing where a non-reasoning model would just handle everything. I wonder if disabling it would make anything better, but likely not if the model is already trained to do it.
CoT works really well in qwen3.8-flash-next. I think a lot of the problems obversed are literal skill issue: dumping huge system prompts/skills that prime it over some broad scope, when the request is narrow.
The system prompt should be considered a starting point only if you want reusable intelligence and focus. It should not be a grab bag of tools and PR style guides, etc...
Grab your agent and inspect its prompt.
Re: How well do agents use test/verification techniques?
#70I think this article demonstrates a broader problem with constraint following in LLMs. Agents often satisfy the most obvious and easily verifiable part of a task, but lose track of the constraint that actually determines success. Testing makes this especially clear: “use fuzzing” turns into generating random bytes, and “use formal verification” into proving a property that isn’t particularly useful. Formally, the tec…
> the problem may be less about knowing a particular technique and more about the agent’s ability to keep its actual goal as the primary constraint.
I think the limiting factor is often the human's ability to articulate the goal and hold the agent accountable.
The working memory of technical people and the way they communicate often seems to prioritize how they're doing a task and not why they're doing it. So when they talk to you about their problem, they give you the human equivalent of modem noises and stack traces. This is such a common problem on tech support and Q&A forums like StackOverflow that it spawned its own terminology and website: https://xyproblem.info/
The agents don't respond to XY problems any better than a colleague does, and usually much worse.
An effective colleague asks why, and we need a similar relationship between the agent and its human supervisor.