Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Is a Harness?

earendil.com

Post reply on HN
Print view View original

111–120 of 200 posts

Re: What Is a Harness?

#111

Post by jbstack »

grim_io wrote: post #45
theturtletalks wrote: post #3

Harnesses are the next frontier. If LLMs are electricity, harnesses are the “electronics.” Right now, it’s like an AC vs DC between Claude and ChatGPT, but once that settles, the harnesses will be the actual value providers. And Pi is the best harness because of the amazing extension system. You can build extensions that turn Pi into a stock trader, software factory, anything. I tried switching to another harness but…

I don't think so. What I can see is a world where we end up with a Chromium-shaped harness, a fully featured standard implementation everyone builds against, because doing every single thing yourself would be crazy. The antithesis to Pi, if you will.

I disagree with this. Unlike training models (which requires huge compute), harness development is available to anyone with an editor and ideas. That means that solo devs and small startups can still make meaningful progress.

Also, having only a "standard implementation" makes no sense for a harness. A standard implementation would need to try to be as good as possible at all things. But you'd often want a specialised harness designed for exactly your use case.

Re: What Is a Harness?

#114

Post by Syntaf »

newsomix9xl wrote: post #72
Syntaf wrote: post #62

I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience. We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents. We paired this with skills initially, but found that the way folks built skil…

Can you post a generic version of code for this somewhere (e.g. codeberg or whatever)? I find your description intriguing but I'd like to see it to make sure I understand it.

Sorry I can't share what we're doing here directly!

I will however say that this page alone does a pretty good job of illustrating what an agent harness might look like: https://docs.agno.com/tools/overview

* System prompt

* Tool calls

* Model definition

Everything else (guards / etc) can just exist as code abstractions between the agent layer and the tool layer.

Re: What Is a Harness?

#115

Post by tedggh »

I feel I’ve been left behind because all I use is Agents.md / Claude.md and Project.md. No skills, no tools. Everything I want my agent to do is specified in those two files. I also have a particular directory and file structure for things like scripts and sources which is indexed in the mds.

Re: What Is a Harness?

#117

Post by anon7000 »

tedggh wrote: post #115

I feel I’ve been left behind because all I use is Agents.md / Claude.md and Project.md. No skills, no tools. Everything I want my agent to do is specified in those two files. I also have a particular directory and file structure for things like scripts and sources which is indexed in the mds.

Fairly sure that your agent is using tools like “edit file” behind the scenes. The post is describing the fundamentals of how agent harnesses work, models don’t exactly have a way to edit files just sending an API request to and forth the provider in a chatbox.

Re: What Is a Harness?

#118

Post by qarl2 »

grey-area wrote: post #104
qarl2 wrote: post #84

Earlier quoted context omitted.

> presuppose competence in LLMs which simply doesn’t exist. Then how do you explain the wild success at using them for development?

Guided by humans, code generators which have ingested the worlds’ code and can recognise and generate patterns can be useful tools. I wouldn’t personally qualify it as a wild success as we are early and there are significant downsides. That doesn’t make them intelligent agents which think independently.

Would you like to see some of my own examples of wild success?

I have a system that entirely reverse engineers old arcade games. Creates semantic symbol mappings that were considered impossible just a couple years ago.

Granted, it took me a couple weeks to build the system.

From impossible to a couple weeks in just a couple years.

Would you like to see it or continue to pretend these things don't exist? Your call.

(It's finding the coolest stuff - the anti-tampering hacks they put into the old machines is fascinating.)

Re: What Is a Harness?

#119

Post by bitwize »

A harness is the thing that turns sloppotron spew into action. That's... pretty much it. LLMs are still "just" token prediction algorithms, but you can coax them into outputting things that look like commands. The harness figures out which bits are supposed to be commands, executes them, and feeds the output back into the context of the LLM. They also have prompts of their own to guide LLM behavior.
Post reply on HN