When SE's cannot agree on the meaning of a tool, it indicates the tool is a placeholder for desire.
Do you think there is disagreement about the meaning of "harness" in AI? I would say there is almost unanimous consensus.
What Is a Harness?
141–150 of 200 posts
Re: What Is a Harness?
#142I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience. We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents. We paired this with skills initially, but found that the way folks built skil…
This couldn't have been said a year ago. It's amazing to watch. I have been building harnesses and applying networked agents to various domains since the GPT-3 API came out, and even two years ago, frontier models were just not at acceptable quality to make these harnesses useful. Everything changed overnight near the end of 2025. What will next year hold?
Re: What Is a Harness?
#143Author here. It’s ironic because this post was clearly geared towards non-hackers. But now that we’re here.. the other analogy I considered presenting was: harness = chassis, model = engine, fuel = tokens, agent = car I’m curious what y’all might think and whether that analogy carries more explanatory power
The first analogy that comes to mind, growing out of "harness", is more like harness = harness, model = horse (rather than harness as in climbing harness). I guess you could say that tokens = hay, and agent = horse and cart, from there? Not sure how useful the hay part is but you could observe from the second that there are many different things you could harness a horse to (also a plough, or a coach, or just a saddl…
I wonder if I'm the only person whose first analogy that came to mind was: model = toddler, harness = baby bouncing jumper harness
Re: What Is a Harness?
#144Harnesses are the next frontier. If LLMs are electricity, harnesses are the “electronics.” Right now, it’s like an AC vs DC between Claude and ChatGPT, but once that settles, the harnesses will be the actual value providers. And Pi is the best harness because of the amazing extension system. You can build extensions that turn Pi into a stock trader, software factory, anything. I tried switching to another harness but…
No, DSH is not built on top of pi
Re: What Is a Harness?
#145Earlier quoted context omitted.
> If LLMs are electricity, harnesses are the “electronics.” (...) the harnesses will be the actual value providers. Don't get ahead of yourself. Harnesses are not exactly rocket science and will be a commodity. The real value providers here are the hardware, then the LLM as a distant second, and at a much larger distance the harness.
https://www.latent.space/p/attention-interface Labs are now post-training models with Harness so that Harness now gets absorbed into the weights.
What is being absorbed into weights is tool usage. It is incredibly counterproductive to train models on a specific harness, when instead it can be trained to reason about the tools that it has available to itself and how to best use those tools to accomplish it's goal.
Would you rather hire an engineer that can adopt to your org's prefered tooling, or hire an engineer that can only perform well with their own favorite tools? It's the same thing.
Re: What Is a Harness?
#146Earlier quoted context omitted.
This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask. As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the…
That works for well trod paths, e.g “fix ci” works exceedingly well. “why app slow” obviously doesn’t work because the task is underspecified. But in order to properly specify you either need an experienced engineer who knows how to narrow the problem domain, or you have to provide some template instructions/output formats (e.g, skills) which will invariably never fit the problem perfectly
disturbingly, when I was using antigravity with gemini pro it was actually quite good at working out 'why app slow' types of problems. Maybe I've been lucky but it seems really good at determining why something might be wrong. It may ask for more logging or diagnostics and run for a long time but it was really digging in and making changes or suggestions to solve the problems.
Re: What Is a Harness?
#147Harnesses are the next frontier. If LLMs are electricity, harnesses are the “electronics.” Right now, it’s like an AC vs DC between Claude and ChatGPT, but once that settles, the harnesses will be the actual value providers. And Pi is the best harness because of the amazing extension system. You can build extensions that turn Pi into a stock trader, software factory, anything. I tried switching to another harness but…
I don't think so. What I can see is a world where we end up with a Chromium-shaped harness, a fully featured standard implementation everyone builds against, because doing every single thing yourself would be crazy. The antithesis to Pi, if you will.
It is way too early to tell. If we are comparing to web browsers we are in the early 90s with Netscape, IE, Firefox, etc. I don't even think we are at the point where agent's have a metaphorical JavaScript, we are that early.
The thing that makes everyone build against Chromium is because web browsers are very hard and it is well supported by dev tools like Electron and Playwright.
Harnesses are so easy compared to a web browser, I'm curious what in this world you see that would make building your own harness seem crazy, because I don't see it.
Re: What Is a Harness?
#148[dead]
Re: What Is a Harness?
#149`harness` is a term invented/borrowed to describe a vague vision of applying llm in a more controlled manner, suitable for semi organized, complex work. the sad part is then people shift the attention from the real problem but seeking inspiration from the terminology.
the real problem: is there a general framework to allow llm with limited context size to approach problems, just like a capable newbie integrated into a chaotic code base? how does it work?
I think it's an epistemic problem, and my fork of pi: pie (github.com/wangii/pie)
Re: What Is a Harness?
#150It is a loop of action-thinking-action based on users'input and LLM response.