Damn it was real the whole time. I found Opus 4.7 to holistically underperform 4.6, and especially in how much wordiness there is. It's harder to work with so I just switched back to 4.6 + Kimi K2.6. Now GPT 5.5 is here and it's been excellent so far.
An update on recent Claude Code quality reports
391–400 of 778 posts
Re: An update on recent Claude Code quality reports
#392yesterday CC created a fastapi /healthz endpoint and told me it's the gold standard (with the ending z). today I stopped my max sub and will be trying codex
Re: An update on recent Claude Code quality reports
#393Earlier quoted context omitted.
Boris, wait, wait, wait, Why not use tired cache? Obviously storage is waaay cheaper than recalculation of embeddings all the way from the very beginning of the session. No matter how to put this explanation — it still sounds strange. Hell — you can even store the cache on the client if you must. Please, tell me I’m not understanding what is going on.. otherwise you really need to hire someone to look at this!)
I assume they are already storing the cache on flash storage instead of keeping it all in VRAM. KV caches are huge - that’s why it’s impractical to transfer to/from the client. It would also allow figuring out a lot about the underlying model, though I guess you could encrypt it. What would be an interesting option would be to let the user pay more for longer caching, but if the base length is 1 hour I assume that wo…
I wonder whether prompt caches would be the perfect use case of something like Optane.
It's kept for long enough that it's expensive to store in RAM, but short enough that the writes are frequent and will wear down SSD storage
Re: An update on recent Claude Code quality reports
#394Earlier quoted context omitted.
Another way to think about it might be that caching is part of Anthropic's strategy to reduce costs for its users, but they are now trying to be more mindful of their costs (probably partly due to significant recent user growth as well as plans to IPO which demand fiscal prudence). Perhaps if we were willing to pay more for our subscriptions Anthropic would be able to have longer cache windows but IDK one hour seems…
That doesn’t make sense to pay more for cache warming. Your session for the most part is already persisted. Why would it be reasonable to pay again to continue where you left off at any time in the future?
Because it significantly increases actual costs for Anthropic.
If they ignored this then all users who don’t do this much would have to subsidize the people who do.
Re: An update on recent Claude Code quality reports
#395>On March 4, we changed Claude Code's default reasoning effort from high to medium to reduce the very long latency—enough to make the UI appear frozen—some users were seeing in high mode Instead of fixing the UI they lowered the default reasoning effort parameter from high to medium? And they "traced this back" because they "take reports about degradation very seriously"? Extremely hard to give them the benefit of do…
Hey, Boris from the team here. We did both -- we did a number of UI iterations (eg. improving thinking loading states, making it more clear how many tokens are being downloaded, etc.). But we also reduced the default effort level after evals and dogfooding. The latter was not the right decision, so we rolled it back after finding that UX iterations were insufficient (people didn't understand to use /effort to increas…
“after evals and dogfooding” couldn’t have done this before releasing the model? We are paying $200/month to beta test the software for you.
Re: An update on recent Claude Code quality reports
#396>On March 4, we changed Claude Code's default reasoning effort from high to medium to reduce the very long latency—enough to make the UI appear frozen—some users were seeing in high mode Instead of fixing the UI they lowered the default reasoning effort parameter from high to medium? And they "traced this back" because they "take reports about degradation very seriously"? Extremely hard to give them the benefit of do…
Hey, Boris from the team here. We did both -- we did a number of UI iterations (eg. improving thinking loading states, making it more clear how many tokens are being downloaded, etc.). But we also reduced the default effort level after evals and dogfooding. The latter was not the right decision, so we rolled it back after finding that UX iterations were insufficient (people didn't understand to use /effort to increas…
Off topic, but I'm hoping you'll maybe see this. There's been an issue with the VS code extension that makes it pretty much impossible to use (PreToolUse can't intercept permission requests anymore, using PermissionRequest hooks always open the diff viewer and steals focus):
https://github.com/anthropics/claude-code/issues/36286 https://github.com/anthropics/claude-code/issues/25018
Re: An update on recent Claude Code quality reports
#397Earlier quoted context omitted.
> I was never under the impression that gaps in conversations would increase costs nor reduce quality. Both are surprising and disappointing. You didn't do your due diligence on an expensive API. A naïve implementation of an LLM chat is going to have O(N^2) costs from prompting with the entire context every time. Caching is needed to bring that down to O(N), but the cache itself takes resources, so evictions have to…
Yes. It’s perfectly reasonable to expect the user to know the intricacies of the caching strategy of their llm. Totally reasonable expectation.
To some extent I'd say it is indeed reasonable. I had observed the effect for a while: if I walked away from a session I noticed that my next prompt would chew up a bunch of context. And that led me to do some digging, at which point I discovered their prompt caching.
So while I'd agree with your sarcasm that expecting users to be experts of the system is a big ask, where I disagree with you is that users should be curious and actively attempting to understand how it works around them. Given that the tooling changes often, this is an endless job.
Re: An update on recent Claude Code quality reports
#398Earlier quoted context omitted.
They need to keep up with demand, because compute resources are clearly limited. That means they have no choice but to add these features, or things break, or they have to stop taking new customers. All of those options are unacceptable.
They're losing customers because of quality concerns. Pausing development and focusing 100% on quality is how you fix that. That said, that may not have been obvious at all in the Jan/Feb time frame when they got a wave of customers due to ethical concerns.
No. Pausing development does not make compute (you know, physical machines?) appear out of thin air.
Re: An update on recent Claude Code quality reports
#399"On March 26, we shipped a change to clear Claude's older thinking from sessions that had been idle for over an hour, to reduce latency when users resumed those sessions. A bug caused this to keep happening every turn for the rest of the session instead of just once, which made Claude seem forgetful and repetitive. We fixed it on April 10. This affected Sonnet 4.6 and Opus 4.6" This makes no sense to me. I often leav…
Hey, Boris from the Claude Code team here. Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache -- everything but the latest message. The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for u…
You created this issue by setting a timer for cache clearing. Time is really not a dimension that plays any role in how coding agent context is used.
Re: An update on recent Claude Code quality reports
#400yesterday CC created a fastapi /healthz endpoint and told me it's the gold standard (with the ending z). today I stopped my max sub and will be trying codex
To be fair that’s a Google convention. Have a look at z-pages