We should discuss cache performance if we haven't already. That 33k tokens may be a cache hit (I am not certain it's automatically a cache hit) but after the first call, it should certainly be a cache hit. Cache hit tokens are billed at 1/10th the price of cache misses. This is quite opaque, but it's necessary when you're asking "is the system prompt worth its stay" if you can save 33k tokens worth of dynamic discove…
Anthropic's cache expires after 1 hour when using subscription endpoints, and for those cached tokens cache reads are free. It's generous (compared to API pricing) but it's not 100% free.
Rather than bake that into the prompt - wouldn’t it be better to just set up a pre commit hook that runs tests and linting?
Maybe, depends on their workflow. In my human workflow, I tend to use commits as checkpoints and then squash before pushing. I'd usually only run time-consuming tests before squash+push. But yes, anything you want to ensure really needs to be a hook. edit: realizing with "precommit" you probably meant a git hook not one in their harness. I'd have written the same response more or less though. :)
Oh yes - definitely the git kind of hook. Also, I always forget that there’s a pre-push hook as well. So you don’t need to do things every commit.
But then you could just be storing up a lot of problems…
You're making the opposite argument. Anthropic is incentivized to use less tokens in Claude Code because people are paying a fixed monthly fee for subscriptions.
Nope, that’s not true, because they want you to pay for the higher subscription bracket.
That strategy only makes sense if there's an abundance of tokens, but that's not the case. AI companies are spending a ton of resources on improving token efficiency because they are all severely GPU constrained. Anthropic instead nudges you to move to a higher tier by setting rate limits.
What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result. If I let the main agent do the same task sequentially, it was no problem at all. I don't know if it's really just communication and orchestration that makes sub agents so inefficient, or…
Same for me. I never use them. I use Fable on highest effort to plan things and then record the plan in tickets. I use Kata, which is CLI and agent oriented, but I suppose Jira or other systems would work too. I tell it to put enough context in each ticket to on-board a fresh coding agent to implement it. Then I just do /goal, telling to to run `kata ready` to get new tickets to work and continue until they're all closed according to acceptance criteria or until they're blocked on actions from me. I need to play around with getting it to switch to smaller models (or spawning 1 subagent) to do ticket implementation and then auto compact after each. Either way, it results in really easy workflows and uses very few tokens compared to the built in subagent flows that doing this completely avoids.
Maybe, depends on their workflow. In my human workflow, I tend to use commits as checkpoints and then squash before pushing. I'd usually only run time-consuming tests before squash+push. But yes, anything you want to ensure really needs to be a hook. edit: realizing with "precommit" you probably meant a git hook not one in their harness. I'd have written the same response more or less though. :)
Oh yes - definitely the git kind of hook. Also, I always forget that there’s a pre-push hook as well. So you don’t need to do things every commit. But then you could just be storing up a lot of problems…
Indeed. That's why I think it depends on the individual's workflow where it should live.
The curiosity is inefficient though. So many times I have to stop the agent and tell it to just fucking write the code and try compiling it. Otherwise it will fill its entire context tracing through the program logic to derive from the code itself whether the thing it is about to do would work. It completely fails to notice it can just… try.
It's tuned for the kinds of tasks where "just try" doesn't get good results. A major complaint with AI code was that AIs struggle with complex codebases, don't respect existing conventions, reinvent functionality multiple times over, etc. So, newer high end AIs are tuned with the "explore/exploit" dial turned towards "explore". You could probably get it to do things "quick and dirty" with prompting, but that, of cour…
Perhaps what is missing is a better memory/caching layer to avoid doing the same for explorations over and over again.
This isn’t limited to large system prompts. Coding-agent harnesses are also becoming more aggressive about using tools, even for trivial requests. In our tests, prompts such as “Hey” or “commit” sometimes triggered 30+ tool calls: https://quesma.com/blog/the-true-cost-of-saying-hi-to-an-ai-... Tokenflation seems very real: the number of tokens consumed by simple tasks keeps increasing.
Why are you asking the LLM to commit? Can’t you do that yourself?
Maybe related to this minimalism, Pi doesn't come with most of the tools an LLM needs to function efficiently or effectively. I get that a blank slate is the paradigm, and you can add whatever you want, but it's too blank IMO.
Oh-my-pi has more tools than claude and opencode, and uses them much more efficiently. my favorites are /collab and the gortex mcp
I tried using omp, and really like the interface, but I found it used tokens much much quicker than the Claude cli. Some simple tasks would use all the session tokens in less than an hour, as where I could get easily get 3-4 hours with Claude. Both set to use opus 4.8 auto effort. I tried tweaking the models for agents down to haiku and sonnet in omp, but didn't notice any real difference in the speed tokens were being used.