Earlier quoted context omitted.
This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask. As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the…
> When I use LLMs I leave things as open as possible. I just give them the information they need and my ask. How do you handle security? Both “internally” against e.g. data loss, I’m assuming via limiting the harness, and “externally”, i.e. stuff like prompt injection risks?
What Is a Harness?
131–140 of 200 posts
Re: What Is a Harness?
#132Author here. It’s ironic because this post was clearly geared towards non-hackers. But now that we’re here.. the other analogy I considered presenting was: harness = chassis, model = engine, fuel = tokens, agent = car I’m curious what y’all might think and whether that analogy carries more explanatory power
I've been building coding harnesses since 2021 and believed in their value for a long time. Harnesses matter a lot, look at what claude did for Anthropic.
Re: What Is a Harness?
#133Re: What Is a Harness?
#134Earlier quoted context omitted.
This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask. As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the…
That works for well trod paths, e.g “fix ci” works exceedingly well. “why app slow” obviously doesn’t work because the task is underspecified. But in order to properly specify you either need an experienced engineer who knows how to narrow the problem domain, or you have to provide some template instructions/output formats (e.g, skills) which will invariably never fit the problem perfectly
Definitely not true and like everyone else is saying, shows how people still underestimate these models.
I have been working on a simple vite + react app lately and commonly ask Gemini/Antigravity to just "improve speeds", "x is running slow, check it out" and have no complaints.
Re: What Is a Harness?
#135I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience. We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents. We paired this with skills initially, but found that the way folks built skil…
I think you've really hit the mark on how the harness should be structured: 1. Guardrails - deterministic, social intelligence, team alignment & accountability 2. Learn by doing 3. make it stupid easy for the agent to research and access data 4. DRY Research supports this. Try picking up some ideas from my harness: https://github.com/rush86999/atom
Re: What Is a Harness?
#136Does anyone have a suggestion for a harness that is good at handoff? When I say handoff, I mean: * handoff from a terminal CLI to webui (on a phone)? * handoff from one team member, to another? * handoff from one communication modality, like writing a prompt in a TUI, to email? * handoff from one model to another, or one provider (openrouter)( to another (llama.cpp) Does such a thing exist? I used to think that a PR…
Re: What Is a Harness?
#137Earlier quoted context omitted.
I've never used Pi but I don't see why you can't use stock codex or claude code for the same purpose, what makes Pi special? I've built plenty of custom harnesses on top of claude code and codex using custom skills or simple markdown instructions and subagents. Never had any issues or limitations with that approach. I do agree that harnesses are going to extend AI capabilities a lot in the next year, but after readin…
For one you can ask Pi to create a TUI extension, so along with the agent interface you can add whatever custom TUI you need, such as portfolio stock tickers, alerts, whatever you want. Many of my harnesses eventually turn into customized UIs around the chat interface.
Re: What Is a Harness?
#138Earlier quoted context omitted.
I think you've really hit the mark on how the harness should be structured: 1. Guardrails - deterministic, social intelligence, team alignment & accountability 2. Learn by doing 3. make it stupid easy for the agent to research and access data 4. DRY Research supports this. Try picking up some ideas from my harness: https://github.com/rush86999/atom
Hi. This is very interesting, could you link to the research? There is a dearth of proper research studies that A/B test what approach is best in terms of harness structure based on repeatable benchmark data with relevant sample uses-cases.
Multi-agent / org (core/agent_radio/, core/fleet_orchestration/): - Stanford Virtual Biotech — bioRxiv 2026.02.23.707551, Zou Lab — https://www.biorxiv.org/content/10.64898/2026.02.23.707551v1 — 37k agents, CSO->scientists->reviewer->re-delegation, Merck external validation of B7-H3 design. Basis for VFS + hierarchy. - Debate or Vote (Choi & Li) — NeurIPS 2025 — https://arxiv.org/abs/2508.17536 — MAD gains = majority vote, not debate (why we didn't build debate)
Sandbox / eval: - DABstep — arXiv:2506.23719 — https://arxiv.org/abs/2506.23719 — 450 real Adyen tasks, justifies code-interpreter + sandbox isolation - Spotlighting — Microsoft Research — https://arxiv.org/abs/2403.14720 — provenance delimiters cut injection ASR 50% -> Governance: - OWASP Top 10 for Agentic Applications 2026 — globally peer-reviewed by 100+ experts, Dec 2025 — https://genai.owasp.org/resource/owasp-top-10-for-agentic-ap... — HIGH. Atom maps 1:1 (Goal Hijack -> match-confidence + oracle, Tool Misuse -> sandbox whitelist/caps, Privilege Abuse -> capability bindings, Memory Poisoning -> verified-episode graduation, etc.) docs/marketing/RESEARCH_NOTES.md:130 - NIST AI Agent Standards Initiative — Feb 17 2026, NIST CAISI — https://www.nist.gov/artificial-intelligence/ai-agent-standa... + RFI summary May 2026 https://www.nist.gov/publications/summary-analysis-responses... — HIGH (US gov standard). Defines the 4 enterprise minimums Atom implements: identification, authorization, access delegation, logging. - Stanford Virtual Biotech — bioRxiv 2026.02.23.707551 — https://www.biorxiv.org/content/10.64898/2026.02.23.707551v1 — CSO -> 4 divisions -> 8 scientists -> reviewer -> re-delegation, no debate, no SFT — HIGH (Stanford Zou lab + Merck external validation). Basis for Atom's fleet hierarchy core/agent_radio/ and why maturity is routing not security. - Spotlighting — Microsoft Research — https://arxiv.org/abs/2403.14720 — HIGH — provenance delimiters cut indirect injection ASR 50% -> https://arxiv.org/abs/2512.00966 + OpenReview — HIGH — intent tracing ASR 100% -> 8.5% on AgentDojo/Mind2Web, basis for sandbox egress allowlist + core/sandbox_tripwire.py
Re: What Is a Harness?
#139Re: What Is a Harness?
#140I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience. We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents. We paired this with skills initially, but found that the way folks built skil…