What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result. If I let the main agent do the same task sequentially, it was no problem at all. I don't know if it's really just communication and orchestration that makes sub agents so inefficient, or…
Spawning a bunch of agents seems to happen randomly. I almost never want this.
As a counterpoint: in a complex project, Fable's "curiosity" may be exactly what you want for an exploration and planning stage - not just for the orchestrator that turns your prompt into different angles with which to explore, but for each subagent whose task is to search the codebase for one of those "angles." If you truly want no stone unturned, letting those subagents spawn their own discoveries, and recursively…
The curiosity is inefficient though. So many times I have to stop the agent and tell it to just fucking write the code and try compiling it. Otherwise it will fill its entire context tracing through the program logic to derive from the code itself whether the thing it is about to do would work. It completely fails to notice it can just… try.
Everything about LLMs is inefficient. They have their benefits but watching them reason over things that are painfully obvious, that they've literally investigated before (before a memory compaction), never take a step back aand be like 'this is going too slow let me look for a better way', etc. is painful.
If you really want a minimal agent that you heavily customize, just skip pi (130+ transitive dependencies on the "minimal" pi-coder package) and write your own. You learn a bunch, and it's not hard. You can even ask another LLM to help you get started.
I wrote my own harness in Emacs and it’s completely ridiculous how well it works. Auto-compact is the only missing feature on my list. Claude‘s approach, if I understand it correctly, invalidates a lot of cached context, and I‘m thinking about a more cache-friendly strategy.
Claude is very cache friendly, however there have been some inconsistencies with non anthropic endpoints that led to cache breakages
I often find myself annoyed when Opus fixes a typo in a comment and decides to run tests, lints and whenever else it can find to run. Often it will start by stashing current changes just to preemptively check if all tests were passing before. And I can blame myself a bit because my rules do say: verify all changes with tests. But as there is that I in AI that is hyped which you’d think means it knows not to put tomat…
> [..] my rules do say: verify all changes with tests I am a bit surprised that you're disappointed that it does exactly what you told it to - people usually have the opposite complaint. If you're using it interactively and watching what it changes, I'd trigger the tests when you think it's needed. And if you want to go more hands-off, why not add try to encode the same nuance you'd use into the rule?
Rather than bake that into the prompt - wouldn’t it be better to just set up a pre commit hook that runs tests and linting?
My opinion is that claude code uses more tokens simply because Anthropic makes more money that way and forces people into their subscriptions. This is supported by the fact that they won't let you use your sub on a different coding agent. I use pi btw.
the amount of system prompt wastage going on in orgs is insane. we identified 400k in annual burn for zero value in just one section of our large company. and the interesting thing about system prompt wastage is its a cost that scales non linearly with subagent use.
The non-linearity is interesting. Is the default behavior for subagents in CC/OpenCode loading the same full system prompt (or AGENTS.md)?
Why don't we have some equivalent of "fork" if we are talking the same context and tokens, you'd think that could all just be loaded into the gpu.
OpenCode, Crush and Pi do have the ability to fork a conversation. But cache reuse is up to the provider and not guaranteed. At some point you need to forward the cache to a more recent checkpoint, and you have a finite (unknown) number of parallel cached chats.
Every subagent send the same ~30k system prompts. If you are using fable/opus, that's easily 30% of a 5-hour window for 7 subagent, before doing any work
If it's always the same prompt, can't they have it pre-cached globally for all?
The system behaviour is totally up to anthropic's discretion. Its current behaviour is verifiable. In claude code, spawn a subagent with
1. Agent("Test")
2. look at your token usage
3. Repeat a few times
I didn't check again as I type this message but am somewhat sure subagent doesn't cache system prompt as of maybe last week
This is interesting, because if I start a fresh session of Claude Code right now and run /context, I see the following: Opus 4.8 (1M context) claude-opus-4-8[1m] 23k/1m tokens (2%) Estimated usage by category System prompt: 3.9k tokens (0.4%) System tools: 13.9k tokens (1.4%) Custom agents: 235 tokens (0.0%) Memory files: 28 tokens (0.0%) Skills: 4.9k tokens (0.5%) Messages: 8 tokens (0.0%) Compact buffer: 3k tokens…
Apologies, you're right - I used imprecise terminology. The entire initial JSON structure that was sent from Claude Claude to the LLM at the start of a session was 162k. This included the system prompt together with a list of tools (some with very extensive explanations), MCP server details, etc. I was simply supporting the article's data - their reported 33k tokens is probably roughly 150-165k.
A lot of people will just add as many tools as they can think of. I don’t think it’s obvious that this costs money.
I still think the best way to build software using LLMs is to copy-paste snippets/files into the chat and manually guide the work. Humans are still the best orchestrators. Yes the human has to now be hyper-focused and juggle various workflows, but the end result (quality of work and throughput of usable code) becomes very good.
I'm surprised most of that isn't cached token usage. It's true that increasing length is a problem on its own because the model needs to attend to it all, but with caching it should be pretty fast anyway. My system prompt is quite large and I haven't noticed much of a generation penalty in the range from 5k to 10k.