Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An update on recent Claude Code quality reports

anthropic.com

Post reply on HN
Print view View original

581–590 of 778 posts

Re: An update on recent Claude Code quality reports

#581

Post by [deleted] »

jwr wrote: post #542
bcherny wrote: post #124

Earlier quoted context omitted.

Hey, Boris from the Claude Code team here. Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache -- everything but the latest message. The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for u…

These controversies erupt regularly, and I hope that you will see a common thing with most of them: you make a decision for your users without informing them. Please fight this hubris. Your users matter. Many of us use your tools for everyday work and do not appreciate having the rug pulled from under them on a regular basis, much less so in an underhanded and undisclosed way. I don't mind the bugs, these will happen…

[deleted]

Re: An update on recent Claude Code quality reports

#582

Post by computably »

miroljub wrote: post #342
computably wrote: post #178

Earlier quoted context omitted.

> I was never under the impression that gaps in conversations would increase costs nor reduce quality. Both are surprising and disappointing. You didn't do your due diligence on an expensive API. A naïve implementation of an LLM chat is going to have O(N^2) costs from prompting with the entire context every time. Caching is needed to bring that down to O(N), but the cache itself takes resources, so evictions have to…

This sounds like a religious cult priest blaming the common people for not understanding the cult leader's wish, which he never clearly stated.

A strange view. The trade-off has nothing to do with a specific ideology or notable selfishness. It is an intrinsic limitation of the algorithms, which anybody could reasonably learn about.

Sure, the exact choice on the trade-off, changing that choice, and having a pretty product-breaking bug as a result, are much more opaque. But I was responding to somebody who was surprised there's any trade-off at all. Computers don't give you infinite resources, whether or not they're "servers," "in the cloud," or "AI."

Re: An update on recent Claude Code quality reports

#583

Post by nubinetwork »

lifthrasiir wrote: post #72

Is it just for me that the reset cycle of usage limits has been randomly updated? I originally had the reset point at around 00:00 UTC tomorrow and it was somehow delayed to 10:00 UTC tomorrow, regardless of when I started to use Claude in this cycle. My friends also reported very random delay, as much as ~40 hours, with seemingly no other reason. Is this another bug on top of other bugs? :-S

My usage got reset yesterday as usual, but it appears it will reset again on Sunday.

Re: An update on recent Claude Code quality reports

#584

Post by UntappedShelf21 »

podnami wrote: post #77

They lost me at Opus 4.7 Anecdotally OpenAI is trying to get into our enterprise tooth and nail, and have offered unlimited tokens until summer. Gave GPT5.4 a try because of this and honestly I don’t know if we are getting some extra treatment, but running it at extra high effort the last 30 days I’ve barely see it make any mistakes. At some points even the reasoning traces brought a smile to my face as it preemptive…

I started using Claude heavily on the 20th after having not used it for a year. Largely Sonnet 4.6, web, cowork and code. Can confidently say it is significantly worse than this time a year ago and regret that my new employer requires we use it, and only it.

Re: An update on recent Claude Code quality reports

#586

Post by uoaei »

sharts wrote: post #382
giwook wrote: post #333

Earlier quoted context omitted.

Another way to think about it might be that caching is part of Anthropic's strategy to reduce costs for its users, but they are now trying to be more mindful of their costs (probably partly due to significant recent user growth as well as plans to IPO which demand fiscal prudence). Perhaps if we were willing to pay more for our subscriptions Anthropic would be able to have longer cache windows but IDK one hour seems…

That doesn’t make sense to pay more for cache warming. Your session for the most part is already persisted. Why would it be reasonable to pay again to continue where you left off at any time in the future?

Exactly, even in the throes of today's wacky economic tides, storage is still cheap. Write the model state immediately after the N context messages in cache to disk and reload without extra inference on the context tokens themselves. If every customer did this for ~3 conversations per user you still would only need a small fraction of a typical datacenter to house the drives necessary. The bottleneck becomes architecture/topology and the speed of your buses, which are problems that have been contended with for decades now, not inference time on GPUs.

Re: An update on recent Claude Code quality reports

#587

Post by thomassmith65 »

xpe wrote: post #514
thomassmith65 wrote: post #471

Earlier quoted context omitted.

I'd rather not speak too poorly of Anthropic, because - to the extent I can bring myself to like a tech company - I like Anthropic. That said, the copy uses "we never intentionally degrade our models" to mean something like "we never degrade one facet of our models unless it improves some other facet of our models" . This is a cop out, because it is what users suspected and complained about. What users want - regardl…

It seems to me you dropped the "gaslighting" claim without owning it. I personally find this frustrating. I prefer when people own up to their mistakes. Like many people, to me, "gaslighting" is just not a term you throw around lightly. Then you shifted to "cop out". (This feels like the motte and bailey.) But I don't think "cop out" is a phrase that works either... Some terms:... The model is the thing that runs inf…

Fair enough. If the comments in question were still editable, I would be happy to replace 'gaslighting' with 'being a bit slippery' or something less controversial.

No worries about 'sounding preachy'; it's a good thing people want to uphold the sobriety that makes HN special.

Re: An update on recent Claude Code quality reports

#589

Post by looshch »

bcherny wrote: post #124
6keZbCECT2uB wrote: post #61

"On March 26, we shipped a change to clear Claude's older thinking from sessions that had been idle for over an hour, to reduce latency when users resumed those sessions. A bug caused this to keep happening every turn for the rest of the session instead of just once, which made Claude seem forgetful and repetitive. We fixed it on April 10. This affected Sonnet 4.6 and Opus 4.6" This makes no sense to me. I often leav…

Hey, Boris from the Claude Code team here. Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache -- everything but the latest message. The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for u…

> We tried a few different approaches to improve this UX

how about acknowledging that you fucked up your own customers’ money and making a full refund for the affected period?

> Educating users on X/social

that is beyond me

ты не Борис, ты максимум борька

Post reply on HN