Build your own vulnerability harness

Post by a3w » Fri, Jul 10, 2026, 10:16 AM UTC

xeyownt wrote: post #19

alberth wrote: post #12

Dumb question: Can someone help me understand whag it means to have a “harness” for agents (and why you need one)? I’ve done some reading on the topic and am just not getting it.

The harness is the tool you use to communicate with the model. It can be as basic as a simple conversation tool (like ChatGPT back in the day), or more advanced like GitHub Copilot, Claude Code, etc. Advanced harnesses will give precise instructions to the model to tell him what tooling / skills / MCPs / etc are available, and will orchestrate all the actions around the model (like starting subagents, calling externa…

Post by jerf » Fri, Jul 10, 2026, 1:55 PM UTC

saithound wrote: post #10

jerf wrote: post #3

This whole vulnerability thing is the first time I'm really feeling like AI tech is a shakedown more than a value-add to my job. So I have to take multiple different frontier models, burn tokens constantly scanning all my code bases with complicated harnesses that eat tokens by the hundreds of millions, burn tokens cross-checking the cross-checks of the cross-checks in some eight-phase process made out of non-determi…

> What happens to the whole AI value proposition when instead of it being a way to pump out lines of code for crazy cheap it becomes a way for each line of code to become vastly more expensive than it was before? I would consider that a good thing. Businesses should not able to make money by writing terrible, insecure code. Anybody who prevents them from doing so surely deserves to capture the additional value create…

Post by throw1234567891 » Fri, Jul 10, 2026, 9:41 PM UTC

jerf wrote: post #24

throw1234567891 wrote: post #13

Earlier quoted context omitted.

> burn tokens constantly Only if you choose to use the most expensive option.

Go ahead and ask Sonnet to do a security scan of your code. You can't afford to be using Sonnet if your opponents are using Fable.