Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An update on recent Claude Code quality reports

anthropic.com

Post reply on HN
Print view View original

541–550 of 778 posts

Re: An update on recent Claude Code quality reports

#541

Post by ElFitz »

ElFitz wrote: post #510
szmarczak wrote: post #421

Earlier quoted context omitted.

> they control the system prompt They control the default system prompt. You can change it if you want to. > They can pick the default reasoning effort Don't see how it's an obstacle in allowing third party wrappers. > They can decide what to keep and what to throw out That's actually a good point. However I still don't think it's an obstacle. If third party wrappers were bad, people simply wouldn't be using them.

Evidently, all these things you just dismissed matter, else all the changes I quoted from the original post wouldn’t have affected anyone, or half as many people, or half as much. Anthropic wouldn’t have had any complaints to investigate, the article promoting this entire thread wouldn’t exist, and we wouldn’t be having this very conversation. Defaults matter . A large share of people never change them (status quo bi…

Edit: the article prompting this entire thread.

Re: An update on recent Claude Code quality reports

#542

Post by jwr »

bcherny wrote: post #124
6keZbCECT2uB wrote: post #61

"On March 26, we shipped a change to clear Claude's older thinking from sessions that had been idle for over an hour, to reduce latency when users resumed those sessions. A bug caused this to keep happening every turn for the rest of the session instead of just once, which made Claude seem forgetful and repetitive. We fixed it on April 10. This affected Sonnet 4.6 and Opus 4.6" This makes no sense to me. I often leav…

Hey, Boris from the Claude Code team here. Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache -- everything but the latest message. The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for u…

These controversies erupt regularly, and I hope that you will see a common thing with most of them: you make a decision for your users without informing them.

Please fight this hubris. Your users matter. Many of us use your tools for everyday work and do not appreciate having the rug pulled from under them on a regular basis, much less so in an underhanded and undisclosed way.

I don't mind the bugs, these will happen. What I do not appreciate is secretly changing things that are likely to decrease performance.

Re: An update on recent Claude Code quality reports

#543

Post by rzk »

puppystench wrote: post #138

The Claude UI still only has "adaptive" reasoning for Opus 4.7, making it functionally useless for scientific/coding work compared to older models (as Opus 4.7 will randomly stop reasoning after a few turns, even when prompted otherwise). There's no way this is just a bug and not a choice to save tokens.

They are forcing users to use adaptive thinking now and deprecating thinking.type: "enabled" and budget_tokens. But the web interface (claude.ai), does not support specifying the effort parameter.

Re: An update on recent Claude Code quality reports

#544

Post by ElFitz »

cmrdporcupine wrote: post #238
ElFitz wrote: post #129

Earlier quoted context omitted.

Because then they lose vertical integration and the extra ability it grants to tune settings to reduce costs / token use / response time for subscription users. Or improve performance and efficiency, if we’re generous and give them the benefit of the doubt. It makes sense, in a way. It means the subscription deal is something along the lines of fixed / predictable price in exchange for Anthropic controlling usage pat…

They gained that ability to tune settings and then promptly used it in a poor way and degraded customer experience.

That’s what we see.

It may be (but I wouldn’t know) that some of other changes not covered here reduced costs on their side without impacting users, improving the viability of their subscription model. Or maybe even improved things for users.

I’d really appreciate more transparency on this, and not just when things fail.

But I’ve learned my lesson. I’ve been weening off Claude for a few weeks, cancelled my subscription three weeks ago, let it expire yesterday, and moved to both another provider and a third-party open source harness.

Re: An update on recent Claude Code quality reports

#545

Post by voxgen »

cmenge wrote: post #401

Bit surprised about the amount of flak they're getting here. I found the article seemed clear, honest and definitely plausible. The deterioration was real and annoying, and shines a light on the problematic lack of transparency of what exactly is going on behind the scenes and the somewhat arbitrary token-cost based billing - too many factors at play, if you wanted to trace that as a user you can just do the work you…

Some of the flak is that issues are often only acknowledged once a fix is in place, and the partial fixes are presented as if they solve the whole problem.

The near-instant transition from "there is no problem" to "we already fixed the problem so stop complaining" is basically gaslighting. (Admittedly the second sentiment comes more from the community, but they get that attitude after taking the "we fixed all the problems" posts at face value.)

Re: An update on recent Claude Code quality reports

#546

Post by tomaskafka »

jwr wrote: post #542
bcherny wrote: post #124

Earlier quoted context omitted.

Hey, Boris from the Claude Code team here. Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache -- everything but the latest message. The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for u…

These controversies erupt regularly, and I hope that you will see a common thing with most of them: you make a decision for your users without informing them. Please fight this hubris. Your users matter. Many of us use your tools for everyday work and do not appreciate having the rug pulled from under them on a regular basis, much less so in an underhanded and undisclosed way. I don't mind the bugs, these will happen…

While I hate all the gaslighting Anthropic seems to do recently (and the fact that their harness broke the code quality, while they forbid use of third party harnesses), making decisions for users is what UX is.

See also the difference between eg. MacOS (with large M, the older good versions) and waiting for "Year of linux on desktop".

I don't think the issue is making decisions for users, but trying to switch off the soup tap in the all-you-can-eat soup bar. Or, wrong business model setting wrong incentives to both sides.

Re: An update on recent Claude Code quality reports

#547

Post by voxgen »

bityard wrote: post #85

My hypothesis is that some of this a perceived quality drop due to "luck of the draw" where it comes to the non-deterministic nature of VM output. A couple weeks ago, I wanted Claude to write a low-stakes personal productivity app for me. I wrote an essay describing how I wanted it to behave and I told Claude pretty much, "Write an implementation plan for this." The first iteration was _beautiful_ and was everything…

I have found Claude to be especially unpredictable. I've mostly switched to GPT-5.4 now - although it's slightly less capable, it's massively more reliable.

Re: An update on recent Claude Code quality reports

#548

Post by Barbing »

Nevermark wrote: post #539
websap wrote: post #430

Earlier quoted context omitted.

Does using print() in Python means I need to understand the Kernel? This is an absurd thought.

That might be an absurd comparison, but we can fix that. If you were being charged per character, or running down character limits, and printing on printers that were shared and had economic costs for stalled and started print runs, then: You wouldn’t “need” to understand. The prints would complete regardless. But you might want to. Personal preference. Which is true of this issue to.

>If you were being charged per character, or running down character limits, and printing on printers that were shared and had economic costs for stalled and started print runs,

and the system was being run by some of the planet’s brightest people whose famous creation is well known to disseminate complex information succinctly,

>then:

You would expect to be led to understand, like… a 1997 Prius.

“This feature showed the vehicle operation regarding the interplay between gasoline engine, battery pack, and electric motors and could also show a bar-graph of fuel economy results.” https://en.wikipedia.org/wiki/Toyota_Prius_(XW10)

Re: An update on recent Claude Code quality reports

#549

Post by meowface »

huksley wrote: post #540

Just add this, it works better than Opus 4.7 vim ~/.claude/settings.json { "model": "claude-opus-4-6", "fastMode": false, "effortLevel": "high", "alwaysThinkingEnabled": true, "autoCompactWindow": 700000 }

Wouldn't xhigh or max work better
Post reply on HN