Earlier quoted context omitted.
The idea here is that your conversation is being swapped out for no good reason. The session is still active, as the subagents continue to make progress. How is this comparable to going to lunch or taking a walk?
Is that actually happening, though?
Show HN: Claude-thermos keeps your Claude session warm for you
41–50 of 91 posts
Re: Show HN: Claude-thermos keeps your Claude session warm for you
#42This is just making it more expensive for everyone else, right? How Claude handles its sessions is none of my business. I'm going to let them do the best they can to provide good service for everyone, and if they can't/won't, I'll switch to a provider that can. Using these massive models is already pretty danged extravagant, I'm not going to demand to be at the front of the queue at all times, too.
From the README: > Detect the danger window. When the main lineage goes idle and a subagent is actively running, the main prefix is at risk of expiring. So, this is not demanding to be at the front of the queue, it's just paying someone to take the place you already had in the queue, when you want to take a leak.
A subagent running doesn't mean that much, though. I often walk away while the subagents are chewing.
Re: Show HN: Claude-thermos keeps your Claude session warm for you
#43They should increase the cache to 10 minutes. 5 is just too low, you can even miss it by taking time to select a response from a question.
10 mins would be amazing
Re: Show HN: Claude-thermos keeps your Claude session warm for you
#44This is just making it more expensive for everyone else, right? How Claude handles its sessions is none of my business. I'm going to let them do the best they can to provide good service for everyone, and if they can't/won't, I'll switch to a provider that can. Using these massive models is already pretty danged extravagant, I'm not going to demand to be at the front of the queue at all times, too.
Eh, I assume Anthropic prices queries somewhere in line with their actual costs. If they don't, and they misalign incentives instead, that's kinda on them.
Re: Show HN: Claude-thermos keeps your Claude session warm for you
#45I thought the cache length was 1 hour, not 5 minutes?
Re: Show HN: Claude-thermos keeps your Claude session warm for you
#46Earlier quoted context omitted.
From the README: > Detect the danger window. When the main lineage goes idle and a subagent is actively running, the main prefix is at risk of expiring. So, this is not demanding to be at the front of the queue, it's just paying someone to take the place you already had in the queue, when you want to take a leak.
A subagent running doesn't mean that much, though. I often walk away while the subagents are chewing.
The defaults seem sensible. This isn't keep-alive-forever by default solution.
Re: Show HN: Claude-thermos keeps your Claude session warm for you
#47Re: Show HN: Claude-thermos keeps your Claude session warm for you
#48Earlier quoted context omitted.
Is that actually happening, though?
Yes https://github.com/izeigerman/claude-thermos/blob/main/src%2...
CC actually prompts Claude about this by default in the ~20k system prompt and instructs it to avoid 300s timeout and to be mindful of the 300s cache expiration.
Re: Show HN: Claude-thermos keeps your Claude session warm for you
#49This is just making it more expensive for everyone else, right? How Claude handles its sessions is none of my business. I'm going to let them do the best they can to provide good service for everyone, and if they can't/won't, I'll switch to a provider that can. Using these massive models is already pretty danged extravagant, I'm not going to demand to be at the front of the queue at all times, too.
How do a few extra requests with the same prefix make it more expensive for everyone else?
The prefix cache is a resource shared by all users. This is basically a tragedy of the commons.
Re: Show HN: Claude-thermos keeps your Claude session warm for you
#50This is just making it more expensive for everyone else, right? How Claude handles its sessions is none of my business. I'm going to let them do the best they can to provide good service for everyone, and if they can't/won't, I'll switch to a provider that can. Using these massive models is already pretty danged extravagant, I'm not going to demand to be at the front of the queue at all times, too.
I agree with you. And sadly, there's nothing we can do about it because it's an arms race. People will try to exploit loopholes to get more benefits and companies will try to prevent exploits--sometimes by making the service worse for normal users.
In this specific case, Anthropic can avoid keeping the cache if it detects this kind of prompt (i.e., if max tokens < some number).