Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An update on recent Claude Code quality reports

anthropic.com

Post reply on HN
Print view View original

491–500 of 778 posts

Re: An update on recent Claude Code quality reports

#491

Post by joefourier »

slashdave wrote: post #373
nickdothutton wrote: post #130

I presume they don't yet have a cohesive monetization strategy, and this is why there is such huge variability in results on a weekly basis. It appears that Anthropic are skipping from one "experiment" to another. As users we only get to see the visible part (the results). Can't design a UI that indicates the software is thinking vs frozen? Does anyone actually believe that?

Compute is limited worldwide. No amount of money can make these compute platforms appear overnight. They are buying time because the only other option is to stop accepting customers.

They would honestly have been better off refusing customers if compute is so limited. Degrading the quality leads to customers leaving in the short term, and ruins their long term reputation.

But in either case, if compute is so limited, they’ll have to compete with local coding agents. Qwen3.6-27B is good enough to beat having to wait until 5PM for your Claude Code limit to reset.

Re: An update on recent Claude Code quality reports

#492

Post by bashtoni »

The Claude Code experience is still pretty bad after upgrading. I often see

  Error: claude-opus-4-7[1m] is temporarily unavailable, so auto mode cannot determine the safety of Bash right now. Wait briefly and then try this action again. If it keeps failing, continue with other tasks that don't require this action and come back to it later. Note: reading files, searching code, and other read-only operations do not require the classifier and can still be used.
The only solution is to switch out of auto mode, which now seems to be the default every time I exit plan mode. Very annoying.

Re: An update on recent Claude Code quality reports

#493

Post by computably »

solarkraft wrote: post #246
computably wrote: post #178

Earlier quoted context omitted.

> I was never under the impression that gaps in conversations would increase costs nor reduce quality. Both are surprising and disappointing. You didn't do your due diligence on an expensive API. A naïve implementation of an LLM chat is going to have O(N^2) costs from prompting with the entire context every time. Caching is needed to bring that down to O(N), but the cache itself takes resources, so evictions have to…

I somewhat disagree that this is due diligence. Claude Code abstracts the API, so it should abstract this behavior as well, or educate the user about it.

I would say this is abstracting the behavior.

Re: An update on recent Claude Code quality reports

#494

Post by afro88 »

rfc_1149 wrote: post #196

The third bug is the one worth dwelling on. Dropping thinking blocks every turn instead of just once is the kind of regression that only shows up in production traffic. A unit test for "idle-threshold clearing" would assert "was thinking cleared after an hour of idle" (yes) without asserting "is thinking preserved on subsequent turns" (no). The invariant is negative space. The real lesson is that an internal message-…

Experienced engineers that know the codebase and system well, and with enough time to consider the problem properly would likely consider this case.

But if we're vibing... This is the kind of bug that should make it back into a review agent/skill's instructions in a more generic format. Essentially if something is done to the message history, check there tests that subsequent turns work as expected.

But yeah, you'd have to piss off a bunch of users in prod first to discover the blind spot.

Re: An update on recent Claude Code quality reports

#495

Post by tasoeur »

podnami wrote: post #77

They lost me at Opus 4.7 Anecdotally OpenAI is trying to get into our enterprise tooth and nail, and have offered unlimited tokens until summer. Gave GPT5.4 a try because of this and honestly I don’t know if we are getting some extra treatment, but running it at extra high effort the last 30 days I’ve barely see it make any mistakes. At some points even the reasoning traces brought a smile to my face as it preemptive…

Same here. I was a fervent Claude code user at $200/mo until Opus4.7.

Freezing your IDE version is now a thing of the past, the new reality is that we can't expect agentic dev workflows to be consistent and I see too many people (including myself) getting burned by going the single-provider route.

On one hand I’m glad to finally see anthropic communicate on this but at this point all I have to say is… time to diversify?

Re: An update on recent Claude Code quality reports

#496

Post by verve_rat »

esafak wrote: post #459
elAhmo wrote: post #219

Earlier quoted context omitted.

Its also kinda funny they have to rely on system prompt to control verbosity itself.

It's cheaper than retraining the model.

So? 4.7.1, 4.7.2, etc. makes sense for versioning system prompts.

Re: An update on recent Claude Code quality reports

#498

Post by matheusmoreira »

vyr wrote: post #408
JumpCrisscross wrote: post #179

Earlier quoted context omitted.

> I was never under the impression that gaps in conversations would increase costs The UI could indicate this by showing a timer before context is dumped.

a countdown clock telling you that you should talk to the model again before your streak expires? that's the kind of UX i'd expect from an F2P mobile game or an abandoned shopping cart nag notification

Why not an automated ping message that's cheap for the model to respond to?

Re: An update on recent Claude Code quality reports

#499

Post by panavm »

kamranjon wrote: post #375

This black box approach that large frontier labs have adopted is going to drive people away. To change fundamental behavior like this without notifying them, and only retroactively explaining what happened, is the reason they will move to self-hosting their own models. You can't build pipelines, workflows and products on a base that is just randomly shifting beneath you.

[flagged]

Re: An update on recent Claude Code quality reports

#500

Post by hakanderyal »

tempest_ wrote: post #439
doesnt_know wrote: post #258

Earlier quoted context omitted.

How do you do "due diligence" on an API that frequently makes undocumented changes and only publishes acknowledgement of change after users complain? You're also talking about internal technical implementations of a chat bot. 99.99% of users won't even understand the words that are being used.

I use CC, and I understand what caching means. I have no idea how that works with a LLM implementation nor do I actually know what they are caching in this context.

CC can explain it clearly, which how I learned about how the inference stack works.
Post reply on HN