[dead]
Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
421–430 of 433 posts
Re: Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
#422Earlier quoted context omitted.
Does your detail plan outline exactly what to implement/ lines of code to edit? I found if I get a nice detailed plan that sonnet is good enough to implement. Did you try that before and found fable better at implementing?
Yeah. So the benefit of Fable is that it'll find gaps in the system as well. So logic errors get caught. I have a separate backend and frontend so plumbing issues are also caught by Fable.
Interesting, have you tried fable with making the strict plan though, and seeing if that catches all of logic errors ahead of time, and then just sending sonnet out to make the detailed updates? Or are you finding that by having fable implement it still catches things post detailed plan? So many tests to run and try out, trying to figure out to try out next.
Re: Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
#423[flagged]
Re: Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
#424Earlier quoted context omitted.
right, but the cache retention time is very short for Anthropics LLMs. 5 minutes or 1 hour (with additional costs). So you have to prompt basically non stop to not get a cache eviction. Anthropic even changed this silently: https://www.reddit.com/r/ClaudeAI/comments/1sk3m12/followup_...
A const prompt across all of Anthropic's subscribers could draw from a global cache rather than per-user? Although saying that out loud makes me question it - each per-user chat and growing cache would need eventually to own its own ~contiguous memory block.
If there are no changes to the system message, yes this is possible and also likely done by Anthropic.
When there are additional local MCPs, reusability will be lower.
I think that Anthropic will bill you in any case :D
Re: Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
#425[flagged]
Re: Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
#426Earlier quoted context omitted.
If you really want a minimal agent that you heavily customize, just skip pi (130+ transitive dependencies on the "minimal" pi-coder package) and write your own. You learn a bunch, and it's not hard. You can even ask another LLM to help you get started.
Any tips on how to get started?
The build your own Claude Code track on Codecrafters is free while in beta.
Re: Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
#427What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result. If I let the main agent do the same task sequentially, it was no problem at all. I don't know if it's really just communication and orchestration that makes sub agents so inefficient, or…
[flagged]
Re: Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
#428Earlier quoted context omitted.
Every subagent send the same ~30k system prompts. If you are using fable/opus, that's easily 30% of a 5-hour window for 7 subagent, before doing any work
The shared prompts are all cached so it's a cache read which is like 10x cheaper than a regular prefill
I'm gonna jump in here and say that the way Anthropic's KV cache works (prefix matching) doesnt guarantee this at all if the subagents are all put into flight at close to the same time. There's nothing in the agent SDK that allows you to instance more than one agent simultaneously, and the docs don't say anything special about KV cache use by agents in this scenario.
You could probably engineer your own harness or workflow to warm the cache by running a single warmup agent, but this is not at all how the tools behave out of the box. It's mostly a cost optimizaiton, so Anthropic really don't have any incentive to do it proactively, and if they did, surely they would want to tell their customers how much they were saving?
Re: Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
#429Earlier quoted context omitted.
Compared to the primary agent, maybe. But it's highly unlikely that all the agents have different tools and system prompts than each other, and those account for the bulk of the context per the post.
Depends on if they are launched serially or in parallel then.
There's no single call that can launch multiple event loops at the same time, so there really isn't a notion of serial or parallel; it's always going to be a simple question of how the first model calls made by each hit the backend. I highly doubt that a bunch of agents created within milliseconds of each other are going to get cache hits. It's more likely they will spin up in parallel. I strongly suspect if Anthropic were trying to deduplicate their inflight requests for cache efficiency, they'd at least want to tell you about it.
Re: Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
#430Earlier quoted context omitted.
Depends on if they are launched serially or in parallel then.
The system prompt and available tools would likely only change for different agent types. So how they're launched probably doesn't matter. I say this as I don't actually know how Clade Code does this, since it's not open source, but I fail to see why two agents doing the same thing launched at different times would have different tools and system prompts.
I think you are right and most people here are overthinking it.
If you make a bunch of identical API calls quickly, before the initial request completes its cache write, you will have cache misses.
If you make a single API call to warm the cache, then make a bunch of subsequent identical requests, you will have cache hits.
The optimization for this sort of thing is to deduplicate in-flight requests, but i've seen no evidence that is being done; the better design pattern would be an api or tool call that explicitly launches multiple agents after a shared prompt processing stage