An update on recent Claude Code quality reports • Hacker News Board

Re: An update on recent Claude Code quality reports

#641

Post by foobarbecue » Fri, Apr 24, 2026, 12:21 PM UTC

bcherny wrote: post #124

6keZbCECT2uB wrote: post #61

"On March 26, we shipped a change to clear Claude's older thinking from sessions that had been idle for over an hour, to reduce latency when users resumed those sessions. A bug caused this to keep happening every turn for the rest of the session instead of just once, which made Claude seem forgetful and repetitive. We fixed it on April 10. This affected Sonnet 4.6 and Opus 4.6" This makes no sense to me. I often leav…

Hey, Boris from the Claude Code team here. Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache -- everything but the latest message. The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for u…

Hi Boris! Wanted to let you know that I find those ads with you saying "now when you code, you use an agent" obnoxious because of that incorrect statement. I have no interest in slop coding. I find it way more ergonomic and effective to use code to tell a machine precisely what to do than to use English to tell it vaguely. I hate that your ad is misleading so many non-coders, who will actually believe your lie that nobody codes anymore. Probably doesn't help that YouTube was playing it as an interruption in every video I watched. I probably saw it 100 times and was getting to the "throw the remote at the tv" stage XD.

Re: An update on recent Claude Code quality reports

#642

Post by bavell » Fri, Apr 24, 2026, 12:42 PM UTC

bontaq wrote: post #504

jannyfer wrote: post #390

Earlier quoted context omitted.

I’m not sure that it’s O(N) with caching but this illustrates the N^2 part: https://blog.exe.dev/expensively-quadratic

If there was an exponential cost, I would expect to see some sort of pricing based on that. I would also expect to see it taking exponentially longer to process a prompt. I don't believe LLMs work like that. The "scary quadratic" referenced in what you linked seems to be pointing out that cache reads increase as your conversation continues? If I'm running a database keeping track of a conversation, and each time it w…

> I would also expect to see it taking exponentially longer to process a prompt. I don't believe LLMs work like that.

Try this out using a local LLM. You'll see that as the conversation grows, your prompts take longer to execute. It's not exponential but it's significant. This is in fact how all autoregressive LLMs work.

Re: An update on recent Claude Code quality reports

#643

Post by jghn » Fri, Apr 24, 2026, 12:57 PM UTC

troupo wrote: post #513

Octoth0rpe wrote: post #438

Earlier quoted context omitted.

There are general users of the average SaaS, and there are claude code users. There's no doubt in my mind that our expectations should be somewhat higher for CC users re: memory. I'm personally not completely convinced that cache eviction should be part of their thought process while using CC, but it's not _that_ much of a stretch.

Anthropic literally advertises long sessions, 1M context, high reasoning etc. And then their vibe-coders tell us that we are to blame for using the product exactly as advertised: https://x.com/lydiahallie/status/2039800718371307603 while silently changing how the product works. Please stop defending hapless innocent corporations.

This oversells how obfuscated it is. I'm far from a power user, and the opposite of a vibe coder. Yet I noticed the effect on my own just from general usage. If I can do it, anyone can do it.

Re: An update on recent Claude Code quality reports

#644

Post by jghn » Fri, Apr 24, 2026, 12:59 PM UTC

abustamam wrote: post #412

jghn wrote: post #397

Earlier quoted context omitted.

To some extent I'd say it is indeed reasonable. I had observed the effect for a while: if I walked away from a session I noticed that my next prompt would chew up a bunch of context. And that led me to do some digging, at which point I discovered their prompt caching. So while I'd agree with your sarcasm that expecting users to be experts of the system is a big ask, where I disagree with you is that users should be c…

> users should be curious and actively attempting to understand how it works Have you ever talked with users? > this is an endless job Indeed. If we spend all our time learning what changed with all our tooling when it changes without proper documentation then we spend all our working lives keeping up instead of doing our actual jobs.

> Have you ever talked with users?

I believe if one were to read my post it'd have been clear that I *am* a user.

This *is* "hacker" news after all. I think it's a safe assumption that people sitting here discussing CC are an inquisitive sort who want to understand what's under the hood of their tools and are likely to put in some extra time to figure it out.

Re: An update on recent Claude Code quality reports

#645

Post by [deleted] » Fri, Apr 24, 2026, 1:09 PM UTC

beering wrote: post #339

robeym wrote: post #228

Earlier quoted context omitted.

What's your workflow like? I'd be curious to test OpenAI out again but Claude Code is how I use the models. Does it require relearning another workflow?

Isn’t it bascially the same thing? You type what you want into the input box and it does what you ask for.

[deleted]

Re: An update on recent Claude Code quality reports

#646

Post by fragmede » Fri, Apr 24, 2026, 1:14 PM UTC

beering wrote: post #339

robeym wrote: post #228

Earlier quoted context omitted.

What's your workflow like? I'd be curious to test OpenAI out again but Claude Code is how I use the models. Does it require relearning another workflow?

Isn’t it bascially the same thing? You type what you want into the input box and it does what you ask for.

Claude code can be configured with custom /slash commands and other details that don't necessarily transfer over to codex. /remote-control in cc is really great for walking away from my computer and continuing from my phone, for instance.

Re: An update on recent Claude Code quality reports

#647

Post by time0ut » Fri, Apr 24, 2026, 1:15 PM UTC

podnami wrote: post #77

They lost me at Opus 4.7 Anecdotally OpenAI is trying to get into our enterprise tooth and nail, and have offered unlimited tokens until summer. Gave GPT5.4 a try because of this and honestly I don’t know if we are getting some extra treatment, but running it at extra high effort the last 30 days I’ve barely see it make any mistakes. At some points even the reasoning traces brought a smile to my face as it preemptive…

Opus 4.7 via code has been inconsistent for me. Sometimes, it feels like working with a brilliant collaborator and is as good as 4.5 and 4.6 were. Other times, it takes dumb and lazy short cuts. It can be quite frustrating. Its response when I tell it it did something wrong is often to write a memory... which is then does not always read. The inconsistency isn't due to session length or age either. These are all new sessions. I feel like sometimes, I get routed do a dumber model or some other hidden setting is applied.

Re: An update on recent Claude Code quality reports

#648

Post by Eridrus » Fri, Apr 24, 2026, 1:18 PM UTC

shrx wrote: post #585

Eridrus wrote: post #460

Earlier quoted context omitted.

Yes. Caching is a cost optimization not a response quality metric.

But it still degrades performance.

It's unfortunate that the word performance is overloaded and ML folks have a specific definition..that isn't what the rest of CS uses, but I understand Anthropic to mean response quality when they say this and not any other dimension you could measure performance on.

You can argue they're lying, but I think this is just folks misunderstanding what Anthropic is saying.

Re: An update on recent Claude Code quality reports

#649

Post by andrewingram » Fri, Apr 24, 2026, 1:23 PM UTC

bavell wrote: post #638

andrewingram wrote: post #631

Earlier quoted context omitted.

This points to a fairly fundamental mismatch between the realities of running an LLM and the expectations of users. As a user, I _expect_ the cost of resuming X hours/days later to be no different to resuming seconds or minutes later. The fact that there is a difference, means it's now being compensated for in fairly awkward ways -- none of the solutions seem good, just varying degrees of bad. Is there a more fundame…

> As a user, I _expect_ the cost of resuming X hours/days later to be no different to resuming seconds or minutes later. As an informed user who understands his tools, I of course expect large uncached conversations to massively eat into my token budget, since that's how all of the big LLM providers work. I also understand these providers are businesses trying to make money and they aren't going to hold every convers…

I'd hazard a guess that there's a large gulf between proportion of users who know as much as you, and the total number using these tools. The fact that a message can perform wildly differently (in either cost, or behaviour if using one of the mitigations) based on whether I send it at t vs t+1 seems like a major UX issue, especially given t is very likely not exposed in the UI.

Re: An update on recent Claude Code quality reports

#650

Post by theshrike79 » Fri, Apr 24, 2026, 1:25 PM UTC

tikkabhuna wrote: post #520

jeremyjh wrote: post #394

Earlier quoted context omitted.

Because it significantly increases actual costs for Anthropic. If they ignored this then all users who don’t do this much would have to subsidize the people who do.

I’m coming at this as a complete Claude amateur, but caching for any other service is an optimisation for the company and transparent for the user. I don’t think I’ve ever used a service and thought “oh there’s a cache miss. Gotta be careful”. I completely agree that it’s infeasible for them to cache for long periods of time, but they need to surface that information in the tools so that we can make informed decision…

Think of it like this: Anthropic has to keep a full virtual machine running just for you. How long should it idle there taking resources when you only pay a static monthly fee and not hourly?

They have a limited number of resources and can’t keep everyone’s VM running forever.