HarnessTax: How Much Does the Harness Matter for Coding Agents?

#1

Post by matt_d » Wed, Sep 16, 2026, 10:10 PM UTC

HarnessTax: How Much Does the Harness Matter for Coding Agents?
harnesstax.github.io

Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?

#2

Post by ed_mercer » Thu, Sep 17, 2026, 12:32 AM UTC

Does this extend to open models like GLM 5.3? This would mean that simply changing the harness to Pi reduces cost in half?

Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?

#3

Post by jswelker » Thu, Sep 17, 2026, 12:35 AM UTC

Much of the extra weight of Claude Code and Codex harness are (ostensibly?) for security and alignment purposes. Whether they are effective is an open question, but leaving those dimensions out and calling it a tax is disingenuous, just turning insecurity into a negative externality.

"Why pay the waste disposal tax? Dumping into the ocean is free!"

Pi actively omits any sort of guardrails and sandboxing in the name of speed and simplicity, so it is not shocking that it is faster and simpler.

Doubling the cost of something in the name of vague security is standard operating procedure for big enterprises, maybe even quite cheap.

Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?

#4

Post by Supermancho » Thu, Sep 17, 2026, 12:43 AM UTC

The term "harness" here is being overloaded for the term "agent", which is worrying. Putting that aside, there are many factors that matter. The "harness" context, the execution pattern (parallel vs sequential), the ability to delegate to other models, etc.

Optimal harnesses use concurrent execution + subagents and are not stuck on one model. Cost and performance are impacted GREATLY by these tactics, regardless of the native agent context (instruction). This kind of single-harness analysis is shallow and misleading, although the finding that "Provider-specific optimization does not guarantee the best pairing" is probably correct, depending on how you measure.

It is a starting point.

Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?

#5

Post by robinpie » Thu, Sep 17, 2026, 1:24 AM UTC

claude code feels mildly shitty to use in the way that every other vibe-coded-project-got-out-of-hand project does, which is like, not that bad, but it's fucking ridiculous for a 2 trillion dollar company's main companion product

Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?

#6

Post by shshsjsj » Thu, Sep 17, 2026, 1:35 AM UTC

jswelker wrote: post #3

Much of the extra weight of Claude Code and Codex harness are (ostensibly?) for security and alignment purposes. Whether they are effective is an open question, but leaving those dimensions out and calling it a tax is disingenuous, just turning insecurity into a negative externality. "Why pay the waste disposal tax? Dumping into the ocean is free!" Pi actively omits any sort of guardrails and sandboxing in the name o…

[dead]

Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?

#7

Post by mikert89 » Thu, Sep 17, 2026, 1:51 AM UTC

As the model gets smarter, you need to tell it less

Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?

#8

Post by appbeforelunch » Thu, Sep 17, 2026, 1:52 AM UTC

[dead]

Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?

#9

Post by NewJazz » Thu, Sep 17, 2026, 1:55 AM UTC

mikert89 wrote: post #7

As the model gets smarter, you need to tell it less

As the black box gets bigger, you need to stop expressing yourself even more.

Re: HarnessTax: How Much Does the Harness Matter for Coding Agents?

#10

Post by roywiggins » Thu, Sep 17, 2026, 2:04 AM UTC

ed_mercer wrote: post #2

Does this extend to open models like GLM 5.3? This would mean that simply changing the harness to Pi reduces cost in half?

I've found the experience of using Pi with local models feels a lot snappier than both OpenCode or Claude Code.