11–20 of 433 posts
And pi agent is even less. The entire agent system prompt can be seen here: https://github.com/earendil-works/pi/blob/main/packages%2Fco...
Anthropic wants to produce the best coding agent possible and doesn’t care (is even incentivized) about high costs. Other harnesses have to make trade offs between performance and cost.
> Claude Code 2.1.207 and OpenCode 1.17.18, both pinned to claude-sonnet-4-5 So not only is this article AI-written, but the testing was entirely done by AI, too? I can't see any other reason to use such an old model. > Our traffic passes through a local LLM gateway that wraps requests in its own envelope, a constant we measured at roughly 6,200 tokens with bare calibration requests Why do you need to do calibration…
Cost, mainly. The runs went through a Claude Max subscription rather than metered API billing, and pinning an older stable snapshot kept run-to-run comparisons clean and cheap. The fixed harness payload (system prompt plus tool schemas), so the headline numbers shouldn't change too much.
That said, happy to re-run the matrix on Fable and publish the diff; payload figures should barely move, tool-calling behaviour might.
Gateway:
Meridian (github.com/rynfar/meridian); proxy that bridges the Claude Code SDK to a standard Anthropic endpoint so a Claude Max subscription can drive OpenCode-et-al.
It's the auth route for all agent traffic on the machine, not something built for the benchmark.
Anthropic wants to produce the best coding agent possible and doesn’t care (is even incentivized) about high costs. Other harnesses have to make trade offs between performance and cost.
pi sends 1k (or less) -> https://github.com/earendil-works/pi/blob/main/packages/codi... My $20 sub using gpt 5.6 sol thinking-off lasts for hours using pi.
I recommend that Opencode users try Dynamic Context Pruning as well: https://github.com/Opencode-DCP/opencode-dynamic-context-pru... It works great for long-horizon tasks, and feels like it saves a boatload of tokens.
I'm generally against this context pruning without prompting or details. Sleev is very opaque about how it works and definitely will bust your cache.
This isn’t limited to large system prompts. Coding-agent harnesses are also becoming more aggressive about using tools, even for trivial requests. In our tests, prompts such as “Hey” or “commit” sometimes triggered 30+ tool calls: https://quesma.com/blog/the-true-cost-of-saying-hi-to-an-ai-... Tokenflation seems very real: the number of tokens consumed by simple tasks keeps increasing.
I am forced to use cloude code at work but a good solution is to just use --system-prompt "" and be done with it. I wish they allowed for other harnesses.