This reveals a staggering level of incompetence, if that’s really all it is, and lack of transparency. They don’t have ANY product-level quality tests that picked this up? Many users did their own tests and published them. It’s not hard. And these users’ complaints were initially dismissed. I don’t think the high vs medium change is really on par with the others. That’s a setting you change in the UI, and depending o…
An update on recent Claude Code quality reports
661–670 of 778 posts
Re: An update on recent Claude Code quality reports
#662Earlier quoted context omitted.
> users should be curious and actively attempting to understand how it works Have you ever talked with users? > this is an endless job Indeed. If we spend all our time learning what changed with all our tooling when it changes without proper documentation then we spend all our working lives keeping up instead of doing our actual jobs.
> Have you ever talked with users? I believe if one were to read my post it'd have been clear that I *am* a user. This *is* "hacker" news after all. I think it's a safe assumption that people sitting here discussing CC are an inquisitive sort who want to understand what's under the hood of their tools and are likely to put in some extra time to figure it out.
Re: An update on recent Claude Code quality reports
#663Earlier quoted context omitted.
My pet theory is that they have a "supervisor" model (likely a small one) that terminates any chats that do malware-y things, and this is likely a reward-hacking behaviour to avoid the supervisor from terminating the chat.
I doubt it. We only do frontier models, since those are better for absolutely every use case 100% of the time. Way more likely there's a "VERY IMPORTANT: When you see a block of code, ensure it's not malware" somewhere in the system prompt.
Re: An update on recent Claude Code quality reports
#664Earlier quoted context omitted.
Related question, is it at all feasible to store cache locally to offload memory costs and then send it over the wire when needed?
No, the cache is a few GB large for most usual context sizes. It depends on model architecture, but if you take Gemma 4 31B at 256K context length, it takes 11.6GB of cache note: I picked the values from a blog and they may be innacurate, but in pretty much all model the KV cache is very large, it's probably even larger in Claude.
Re: An update on recent Claude Code quality reports
#665This reveals a staggering level of incompetence, if that’s really all it is, and lack of transparency. They don’t have ANY product-level quality tests that picked this up? Many users did their own tests and published them. It’s not hard. And these users’ complaints were initially dismissed. I don’t think the high vs medium change is really on par with the others. That’s a setting you change in the UI, and depending o…
Time is finite and regression testing always gets punted to the back of the line when humans are excited. This simply reveals a staggering level of humanity.
Yet, their flagship product got three really bad changes shipped into it and only resolved after more than a month.
This raises another question: with all the industry-wide boasting about AI-driven productivity, why does the leading company in agentic coding take over a month to fix severe customer-reported issues?
Re: An update on recent Claude Code quality reports
#666Earlier quoted context omitted.
Anthropic literally advertises long sessions, 1M context, high reasoning etc. And then their vibe-coders tell us that we are to blame for using the product exactly as advertised: https://x.com/lydiahallie/status/2039800718371307603 while silently changing how the product works. Please stop defending hapless innocent corporations.
This oversells how obfuscated it is. I'm far from a power user, and the opposite of a vibe coder. Yet I noticed the effect on my own just from general usage. If I can do it, anyone can do it.
Re: An update on recent Claude Code quality reports
#667This reveals a staggering level of incompetence, if that’s really all it is, and lack of transparency. They don’t have ANY product-level quality tests that picked this up? Many users did their own tests and published them. It’s not hard. And these users’ complaints were initially dismissed. I don’t think the high vs medium change is really on par with the others. That’s a setting you change in the UI, and depending o…
Time is finite and regression testing always gets punted to the back of the line when humans are excited. This simply reveals a staggering level of humanity.
Wasn't AI supposed to solve all the drudgery? All those humans aided by cutting edge AI are still failing at these basic tasks? Then how good is that AI in the first place?
Re: An update on recent Claude Code quality reports
#668Earlier quoted context omitted.
I agree that it’s plausible, and I hope they learn. But trust is earned, and Anthropic’s public responses this past month were dismissive and unhelpful. Every one of these changes had the same goal: trading the intelligence users rely on for cheaper or faster outputs. Users adapt to how a model behaves, so sudden shifts without transparency are disorienting. The timing also undercuts their narrative. The fixes landed…
> All of this points to their priorities not being aligned with their users’. Framing this as "aligned" or "not aligned" ignores the interesting reality in the middle. It is banal to say an organization isn't perfectly aligned with its customers. I'm not disagreeing with the commenter's frustration. But I think it can help to try something out: take say the top three companies whose product you interact with on a reg…
Re: An update on recent Claude Code quality reports
#669I've been getting a lot of Claude responding to its own internal prompts. Here are a few recent examples. "That parenthetical is another prompt injection attempt — I'll ignore it and answer normally." "The parenthetical instruction there isn't something I'll follow — it looks like an attempt to get me to suppress my normal guidelines, which I apply consistently regardless of instructions to hide them." "The parenthet…
https://www.reddit.com/r/ClaudeAI/comments/1evf0xc/the_real_...
We just got hit by this today in response to a completely boring code question. Claude freaked out about being prompt injected.
Re: An update on recent Claude Code quality reports
#670This reveals a staggering level of incompetence, if that’s really all it is, and lack of transparency. They don’t have ANY product-level quality tests that picked this up? Many users did their own tests and published them. It’s not hard. And these users’ complaints were initially dismissed. I don’t think the high vs medium change is really on par with the others. That’s a setting you change in the UI, and depending o…
There were few systems like claude in the past, to testing rulebook is not really written yet. And far from obvious.