[flagged]
What Is a Harness?
101–110 of 200 posts
Re: What Is a Harness?
#102Earlier quoted context omitted.
This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask. As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the…
That works for well trod paths, e.g “fix ci” works exceedingly well. “why app slow” obviously doesn’t work because the task is underspecified. But in order to properly specify you either need an experienced engineer who knows how to narrow the problem domain, or you have to provide some template instructions/output formats (e.g, skills) which will invariably never fit the problem perfectly
I use skills. The skills are not typically "how to perform a task in detail" they are more about what relevant tools and knowledge are required to work in a domain. That is I give the LLM the information it needs about the system but not a sequence of how to accomplish a task. I treat it more like a human and less like a computer.
Re: What Is a Harness?
#103Great example of writing about AI that maintains a human voice. Starting off the post with a picture of the author + nod to real-world experience (climbing) is a reasonably strong “this is not slop” signal.
OK, I agree.
FYI, the pic is Royal Robbins (https://en.wikipedia.org/wiki/Royal_Robbins), who was kind of a hero to some of my climbing buddies back in the 70's.
Re: What Is a Harness?
#104Earlier quoted context omitted.
I’m afraid it’s a terrible metaphor, starting with the fact that LLMs are nothing like electricity, and the relation of harnesses to them is nothing like that of electronics to electricity, save perhaps one is a prerequisite of the other. Harnesses (and the concept of agents before them) presuppose competence in LLMs which simply doesn’t exist.
> presuppose competence in LLMs which simply doesn’t exist. Then how do you explain the wild success at using them for development?
Guided by humans, code generators which have ingested the worlds’ code and can recognise and generate patterns can be useful tools. I wouldn’t personally qualify it as a wild success as we are early and there are significant downsides.
That doesn’t make them intelligent agents which think independently.
Re: What Is a Harness?
#105Author here. It’s ironic because this post was clearly geared towards non-hackers. But now that we’re here.. the other analogy I considered presenting was: harness = chassis, model = engine, fuel = tokens, agent = car I’m curious what y’all might think and whether that analogy carries more explanatory power
The first analogy that comes to mind, growing out of "harness", is more like harness = harness, model = horse (rather than harness as in climbing harness).
I guess you could say that tokens = hay, and agent = horse and cart, from there? Not sure how useful the hay part is but you could observe from the second that there are many different things you could harness a horse to (also a plough, or a coach, or just a saddle) based on your goal.
Re: What Is a Harness?
#106Author here. It’s ironic because this post was clearly geared towards non-hackers. But now that we’re here.. the other analogy I considered presenting was: harness = chassis, model = engine, fuel = tokens, agent = car I’m curious what y’all might think and whether that analogy carries more explanatory power
I'm a climber so I'm biased but I really liked your climbing harness example because of the configuration you're able to easily make to the harness.
Saying the harness is like a car's chassis doesn't work as well for me because the chassis isn't as configurable as a climbing harness for as little work.
Getting deeper into the climbing analogy you can even swap out the harnesses themselves for wildly different climbs. Like using Claude Code with a bunch of agents for medical software (climbing K2 where that extra padding comes in super handy) and pi.dev with a local model for a respectable web project (sport route where you'll be back in a few hours and it's safe to be a little more exposed).
I'm glad your article made HN, and thank you for pi!
Re: What Is a Harness?
#107[dead]
Re: What Is a Harness?
#108The ai hype word for 2026 after agent in 2025 for any LLM powered application. Well kind of, I wouldn't be surprised to see that some things marketed as agents are actually good old deterministic software.
My guess is that everything "reliable" in LLM/agentic-coding comes down to either calls to reliable/deterministic tools or providing well-defined success criteria (such as loads of unit tests) for the LLM to throw its stuff at in "agentic loops" until something sticks.
Re: What Is a Harness?
#109Earlier quoted context omitted.
This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask. As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the…
That works for well trod paths, e.g “fix ci” works exceedingly well. “why app slow” obviously doesn’t work because the task is underspecified. But in order to properly specify you either need an experienced engineer who knows how to narrow the problem domain, or you have to provide some template instructions/output formats (e.g, skills) which will invariably never fit the problem perfectly
> . “why app slow” obviously doesn’t work because the task is underspecified.
Not always. In my case LLM goes to grafana mcp, pulls metrics/traces/cpu profiles. Figures out what is slow and proposes a solution.
Re: What Is a Harness?
#110I feel like harnesses will become massively important for enterprise AI agents.
Right now every tool is shipping some kind of AI agent, but I can’t help but feel that AI agents in large companies will eventually be some kind of internal app with internal MCPs, CLIs, APIs etc.
There might be different harnesses for different use cases that different people have different levels of access to.
This would make sense for the platform/infrastructure engineers who can build a modular harness that a person or team can get access to.
You could have agents team members use locally that have memory enabled for personalization and then agents that anyone can use to ask questions about company context, which wouldn’t personalize things.