When rewinding to an earlier turn, what if that turn is more than 1hr old? Can this cause KV-cache misses compared to continuing the conversation?
Maximizing the value of your Claude Code sessions
81–90 of 206 posts
Re: Maximizing the value of your Claude Code sessions
#82I mean, it feels hard not to laugh at this type of blog post. My cynical interpretation is that this is a type of passing the buck to engineers in enterprise settings ("Stop spending tokens. Did you read the value maximization blog post? It is your fault.") Oh yes, Claude will do all sorts of different things -- it depends on how you use it! You should totally learn all of these little finicky things ... because now…
I'm curious if you also laugh at articles about how to reduce your AWS bill, or how to add indices to Postgres such that you can run it on cheaper hardware.
You can generally understand your AWS cloud usage, and waste can be self evident with their existing tools. Not at all with llms.
A postgres index post is unlikely to reach front page. It's already part if the docs, and should include more context to be read worthy.
They are not equal comparison.
This before the fact that there is no guarantee that a model follows your agent instructions (plenty of easy to reach for research on it), and you also get suggestions by devs at these companies to wipe parts of your model's instructions because the model is better now tm.
If cloud providers change their billing quasi monthly, and if you'd need to fiddle with your indexes every couple of days. I'm not sure we'd be using them as much.
There is interesting information about the inference pipeline, but almost too late to the party (by at least a year), and for which audience? Techies understand in broad strokes the tech if they are interested, normies will definitely not read it.
All that to say, that yes, it's worth having a laugh. If for nothing else, as a release valve for all the problems they create in the real non-VC world.
Anthropic is IPOing in October according to news, you might be interested in investing.
Re: Maximizing the value of your Claude Code sessions
#83Recently I came across the /handoff skill, which I've been using a lot. I find it much better than /compact. Basically: - /handoff file creates a short document with the important context from your current session and maybe next steps as checklist. - You can then start a fresh session with /continue file - You can also hand the work from Claude to ChatGPT, or the other way around. Very useful at time of session limit…
/handoff is the skill I use when I plainly are about to run out of token budget to give to a different harness system. Using this to avoid long sessions is something I will need to try, plus maybe "caveman" mode to cull the verbosity.
Re: Maximizing the value of your Claude Code sessions
#84Earlier quoted context omitted.
Perhaps my enterprise cynicism is not warranted, but my other comments refer to accurate descriptions of reality: Anthropic wants to place their opaque system between you and any computational task that you wish to perform. Do you contest this or think it is not accurate? Why do you think that Anthropic wants fewer tokens inputted and outputted?
Because they sell subscriptions and tokens cost them compute, and their margin lives in the difference between what your subscription pays in and what you cost them in compute. They have also been supply constrained on compute and if users cost them less in compute they can more subscriptions and less customer frustration. I agree they want you to have a subscription. That doesn't mean they aren't aligned with their…
Subscriptions are a very small part of their overall revenue (estimates have been between 5% and 20% based on financial reporting). Enterprise users are charged per-token, and maximal input/output tokens nets them maximal revenue.
Re: Maximizing the value of your Claude Code sessions
#85No RSS feed for their blog.
Re: Maximizing the value of your Claude Code sessions
#86how many times it stays there i think doesnt give the best comparison
if youre working on the same codebase, that cache stays quite relevant, and i dont think they make the case that clearing and reading the same couple files over and over again is cheaper that relying on it already being cached. same with doing some of the same teaching claude the right way to approach changes in that codebase again and again.
what would be nice is pulling back and reusing an earlier part of the cache for the later two tasks, but claude code doesnt make that particularly easy, and using an LLM to pick where to go back to isnt really gonna save much when it reads all the same text again.
Re: Maximizing the value of your Claude Code sessions
#87Is it possible to have some kind of script to keep your cache warm, or auto compact or something. I sometimes just leave some goals or something running before I go to bed or out and I don’t want to pay the cache text when I come back.
If everyone does it they lose the memory savings they're getting by expiring the cache.
which is to say that they set their ttl too short, it should be longer than people spend at lunch
Re: Maximizing the value of your Claude Code sessions
#88Recently I came across the /handoff skill, which I've been using a lot. I find it much better than /compact. Basically: - /handoff file creates a short document with the important context from your current session and maybe next steps as checklist. - You can then start a fresh session with /continue file - You can also hand the work from Claude to ChatGPT, or the other way around. Very useful at time of session limit…
I have been doing this a lot even without a skill, having Fable write a planning document, then spawning an Opus subagent with instructions to strictly follow the plan and report any deviance at the end. It also helps that then the plan is always saved in an md file so any future agent can look at it and see what happened.
Interestingly, this was tackled in this blog post[0] a month ago. They claim that plan files aren't token-efficient, because after reading the plan the workhorse model then reads all the relevant files anyways.
Re: Maximizing the value of your Claude Code sessions
#89Anthropic should build a harness (and model) that smartly takes care of all these points. Not requiring the user to do the manual work. All I see are excuses because they cannot handle the load and enforce strict quotas on users, all while OpenAI constantly resets their quotas.
With Qwen 3.8 27B, we're one step closer to on-device LLMs that can replace subscriptions.
Re: Maximizing the value of your Claude Code sessions
#90Earlier quoted context omitted.
Because they sell subscriptions and tokens cost them compute, and their margin lives in the difference between what your subscription pays in and what you cost them in compute. They have also been supply constrained on compute and if users cost them less in compute they can more subscriptions and less customer frustration. I agree they want you to have a subscription. That doesn't mean they aren't aligned with their…
Subscriptions are a very small part of their overall revenue (estimates have been between 5% and 20% based on financial reporting). Enterprise users are charged per-token, and maximal input/output tokens nets them maximal revenue.
They still want you to hit the cache because their margin is higher on cache hits. That's actual compute they don't have to pay for and they don't have to have capacity for because they are supply limited on the compute side.
And the unit economics need to be there because there are competitors in the space. They can't just skin you on tokens or you'll jump ship.