481–490 of 695 posts
Earlier quoted context omitted.
Why is nobody even asking why that should be an issue? No other text editor shits the bed that way. The whole point of the computer is that it patiently waits for my input.
let me put this way: not your ram, not your cache, not waiting patiently for your input.
Earlier quoted context omitted.
So if I run a test suite or compile my rust program in a sub agent I’m going to get cache misses? Boo.
Sub agents don't have much context and don't stay around for long, so misses in that case are trivial.
I'm afraid the music may be slowly fading at this party, and the lights will soon be turned on. We may very well look back on the last couple years as the golden era of subsidized GenAI compute. For those not in the Google Gemini/Antigravity sphere, over the last month or so that community has been experiencing nothing short of contempt from Google when attempting to address an apparent bait and switch on quota expec…
Indeed. Anthropic is just leading the pack switching to juicy corporate users who are happy to pay thousands per month per dev and leave the fans behind. And now OpenAI is following suit. They lowered significantly the limits for the Plus $20 plan and answered concerns with vague confusing tweets about promotions.
All this is pushed by the fastest rising demand (Codex growing +50% monthly) while having a serious bottleneck building data centers and getting parts (permits, energy, memory, flash, etc).
Users on reddit and Discord are trying to switch to open models or Chinese alternatives. But there's no real replacement.
Earlier quoted context omitted.
> 1hr -> 5min on March 6th This is not accurate. The main agent typically uses a 1h cache (except for API customers, which can enable 1h but it is not on by default because it costs more). Sub-agents typically use a 5m cache.
... so how do API users enable 1hr caching? I haven't found a setting anywhere.
there is env.ENABLE_PROMPT_CACHING_1H_BEDROCK - but that is - as the name says "when using Bedrock"
for the raw API the docs are also clear -> "ttl": "1h" https://platform.claude.com/docs/en/build-with-claude/prompt...
but how to make claude-code send that when paying by API-key? or when using a custom ANTHROPIC_BASE_URL? (requests will contain cache_control, but no ttl!)
Earlier quoted context omitted.
Ya I've had this experience more than a few times recently. I've heard people claiming they are serving quantized models during high loads, but it happens in cursor as well so I don't think it's specific to Anthropics subscription. It could be that the context window has just gotten into a state that confuses the model... But that wouldn't explain why it appears to be temporary... My best guess is this is the result…
Cursor one is back to Claude 4 or 3.5+ at best. Struggles to do things it did effortlessly a few weeks ago. It’s not under load either it’s just fully downgraded. Feels more they’re dialing in what they can get away with but are pushing it very far.
Earlier quoted context omitted.
The evidence is that quotas exist, as seen here, and are low enough that people are hitting them regularly. When was the last time you hit your quota of Google searches? When was the last time you hit your quota of StackOverflow questions? When was the last time you hit your quota of YouTube videos? Any service will rate limit abuse, but if abuse is indistinguishable from regular use from the provider's perspective,…
It's also kind of interesting that they don't think they can do what an economy would normally do in this situation, which is raise prices until supply matches. Shortages generally imply mispricing. There's a lot of angles you take from that as a starting point and I'm not confident that I fully understand it, so I'll leave it to the reader.
if the prices dont keep going down, the pitch falls apart, that you need a specialist to come in and make it work
Earlier quoted context omitted.
> Since Claude Code uses a 1 hour prompt cache window for the main agent, if you leave your computer for over an hour then continue a stale session, it's often a full cache miss. To improve this, we have shipped a few UX improvements (eg. to nudge you to /clear before continuing a long stale session), and are investigating defaulting to 400k context instead I don’t understand this. I frequently have long breaks. I ne…
Makes me wish that shortly before the server-side expiration, we could save the cache on the client-side, indefinitely. But my understanding is that we're talking about ~60GB of data per session, so it sounds unrealistic to do...
But yes, would love to save context/cache such that it can be played back/referred to if needed.
/compact is a little black box that I just have to trust that is keeping the important bits.
Earlier quoted context omitted.
So if I run a test suite or compile my rust program in a sub agent I’m going to get cache misses? Boo.
Sub agents don't have much context and don't stay around for long, so misses in that case are trivial.
Earlier quoted context omitted.
We are taking it seriously, and are continuing to investigate. We are not trusting the metrics.
The quantitative ux research team at Google was created for exactly this problem: a service which became popular before the right metrics existed, meaning metrics need to be derived first, then optimized. We would observe users (irl), read their logs, then generate experiments to improve the behavior as measured by logs, and return to see if the experiment improves irl experiences. There were not many of us and we ar…
Many of the metrics they use are obviously actively user hostile.
Try it out and you will quickly see how much money they‘d really like for your excessive usage.