What Is a Harness? • Hacker News Board

Re: What Is a Harness?

#91

Post by datsci_est_2015 » Sun, Aug 23, 2026, 6:13 PM UTC

layer8 wrote: post #70

conmod278 wrote: post #29

Earlier quoted context omitted.

https://www.latent.space/p/attention-interface Labs are now post-training models with Harness so that Harness now gets absorbed into the weights.

I’d say that harnesses almost by definition are the parts that you want to keep customizable. That won’t get absorbed into the weights.

Depends on your product strategy. If you only care about how your model will be used in the context of a harness (perhaps, specifically the harness that you designed), then the incentive is plainly there to optimize the weights within the context of the harness.

Re: What Is a Harness?

#92

Post by YZF » Sun, Aug 23, 2026, 6:25 PM UTC

Syntaf wrote: post #62

I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience. We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents. We paired this with skills initially, but found that the way folks built skil…

This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask.

As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the model, it fills the context with conflicting instructions, it reduces the ability of the agent to respond to novel situations (and really in real life most situations are going to be novel). If you want to follow a process or a checklist you probably shouldn't use an LLM, or you should use it for some sub-tasks in the checklist/process but something more deterministic to work through the list.

Re: What Is a Harness?

#93

Post by kmansm27 » Sun, Aug 23, 2026, 6:27 PM UTC

From these comments, it seems like people still don't understand what harnesses are... The point is you shouldn't build a harness, you should use a harness and change its system prompt, the tools it has, MCPs it has, give it skills, etc, to make it work for your usecase. You aren't "building a harness on top of pi" if all you're doing is the above. You're just using the harness to connect different things to the LLM.

Re: What Is a Harness?

#94

Post by YZF » Sun, Aug 23, 2026, 6:27 PM UTC

xrd wrote: post #74

Does anyone have a suggestion for a harness that is good at handoff? When I say handoff, I mean: * handoff from a terminal CLI to webui (on a phone)? * handoff from one team member, to another? * handoff from one communication modality, like writing a prompt in a TUI, to email? * handoff from one model to another, or one provider (openrouter)( to another (llama.cpp) Does such a thing exist? I used to think that a PR…

I do this all the time in my workflow. Use any harness. Ask it to create a markdown file with the information required for the handoff. Use that downstream. Keep a "repo" of those markdown files. Are you trying to orchestrate or manage this sort of process?

Re: What Is a Harness?

#95

Post by wonnage » Sun, Aug 23, 2026, 6:44 PM UTC

YZF wrote: post #92

Syntaf wrote: post #62

I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience. We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents. We paired this with skills initially, but found that the way folks built skil…

This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask. As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the…

That works for well trod paths, e.g “fix ci” works exceedingly well. “why app slow” obviously doesn’t work because the task is underspecified. But in order to properly specify you either need an experienced engineer who knows how to narrow the problem domain, or you have to provide some template instructions/output formats (e.g, skills) which will invariably never fit the problem perfectly

Re: What Is a Harness?

#96

Post by itishappy » Sun, Aug 23, 2026, 6:44 PM UTC

xrd wrote: post #74

Does anyone have a suggestion for a harness that is good at handoff? When I say handoff, I mean: * handoff from a terminal CLI to webui (on a phone)? * handoff from one team member, to another? * handoff from one communication modality, like writing a prompt in a TUI, to email? * handoff from one model to another, or one provider (openrouter)( to another (llama.cpp) Does such a thing exist? I used to think that a PR…

The session is "just" the raw chat history in it's entirety (human and agent) and can be disseminated as such. This is what enables swapping between models, you simply send the whole context.

Not sure how others do it, but opencode stores sessions in a sqlite db and you can extract them and share them as needed.

https://opencode.ai/docs/cli/#export

Pro-tip: Building your own extremely minimal harness takes about 15m and is both fun and enlightening. Agents are unsurprisingly quite good at it, but ask them to walk you through it step by step.

Re: What Is a Harness?

#97

Post by ni10c » Sun, Aug 23, 2026, 6:48 PM UTC

Author here. It’s ironic because this post was clearly geared towards non-hackers. But now that we’re here.. the other analogy I considered presenting was:

harness = chassis, model = engine, fuel = tokens, agent = car

I’m curious what y’all might think and whether that analogy carries more explanatory power

Re: What Is a Harness?

#98

Post by phforms » Sun, Aug 23, 2026, 6:48 PM UTC

childofhedgehog wrote: post #2

Clear, relevant, and easy to understand. Thank you for writing this up, I’ll be sharing this link with all my non-tech friends!

I second that. Not using agents myself but trying to get an idea on how this stuff works, so I always wondered what an "harness" even is, since anyone seems to assume that this is common knowledge. Now it is really clear to me!

Re: What Is a Harness?

#99

Post by gmadsen » Sun, Aug 23, 2026, 6:51 PM UTC

wonnage wrote: post #95

YZF wrote: post #92

Earlier quoted context omitted.

This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask. As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the…

That works for well trod paths, e.g “fix ci” works exceedingly well. “why app slow” obviously doesn’t work because the task is underspecified. But in order to properly specify you either need an experienced engineer who knows how to narrow the problem domain, or you have to provide some template instructions/output formats (e.g, skills) which will invariably never fit the problem perfectly

It really doesn’t need to be that much more specified, give it context to the tools and level of analysis you expect then “why app slow” is a reasonable prompt

Re: What Is a Harness?

#100

Post by hombre_fatal » Sun, Aug 23, 2026, 6:56 PM UTC

wonnage wrote: post #95

YZF wrote: post #92

Earlier quoted context omitted.

This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask. As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the…

That works for well trod paths, e.g “fix ci” works exceedingly well. “why app slow” obviously doesn’t work because the task is underspecified. But in order to properly specify you either need an experienced engineer who knows how to narrow the problem domain, or you have to provide some template instructions/output formats (e.g, skills) which will invariably never fit the problem perfectly

I wouldn't agree. Sota models can do self-directed sampling, profiling, benchmarking, read call trees, etc. to give you a report of the app's bottlenecks and then recommend solutions that can be vetted.

I do this constantly.

As the upstream comment points you, you don't need to specify. Sota models are that good. And by being overprescriptive you can accidentally shut off branches that they would've taken, downgrading the quality of their work.