Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An update on recent Claude Code quality reports

anthropic.com

Post reply on HN
Print view View original

691–700 of 778 posts

Re: An update on recent Claude Code quality reports

#691

Post by libraryofbabel »

tedivm wrote: post #680
libraryofbabel wrote: post #664

Earlier quoted context omitted.

To extend your point: it's not really the storage costs of the size of the cache that's the issue (server-side SSD storage of a few GB isn't expensive), it's the fact that all that data must be moved quickly onto a GPU in a system in which the main constraint is precisely GPU memory bandwidth. That is ultimately the main cost of the cache. If the only cost was keeping a few 10s of GB sitting around on their servers,…

That cost that you're talking about doesn't change based on how long the session is idle. No matter what happens they're storing that state and bring it back at some point, the only difference is how long it's stored out of GPU between requests.

Are you sure about that? They charge $6.25 / MTok for 5m TTL cache writes and $10 / MTok for 1hr TTL writes for Opus. Unless you believe Anthropic is dramatically inflating the price of the 1hr TTL, that implies that there is some meaningful cost for longer caches and the numbers are such that it's not just the cost of SSD storage or something. Obviously the details are secret but if I was to guess, I'd say the 5m cache is stored closer to the GPU or even on a GPU, whereas the 1hr cache is further away and costs more to move onto the GPU. Or some other plausible story - you can invent your own!

Re: An update on recent Claude Code quality reports

#693

Post by troupo »

jghn wrote: post #643
troupo wrote: post #513

Earlier quoted context omitted.

Anthropic literally advertises long sessions, 1M context, high reasoning etc. And then their vibe-coders tell us that we are to blame for using the product exactly as advertised: https://x.com/lydiahallie/status/2039800718371307603 while silently changing how the product works. Please stop defending hapless innocent corporations.

This oversells how obfuscated it is. I'm far from a power user, and the opposite of a vibe coder. Yet I noticed the effect on my own just from general usage. If I can do it, anyone can do it.

Here's Anthropic's own Boris Cherny and others telling how great everything is with long sessions and contexts: https://news.ycombinator.com/item?id=47886087

Re: An update on recent Claude Code quality reports

#694

Post by QuantumGood »

bcherny wrote: post #124
6keZbCECT2uB wrote: post #61

"On March 26, we shipped a change to clear Claude's older thinking from sessions that had been idle for over an hour, to reduce latency when users resumed those sessions. A bug caused this to keep happening every turn for the rest of the session instead of just once, which made Claude seem forgetful and repetitive. We fixed it on April 10. This affected Sonnet 4.6 and Opus 4.6" This makes no sense to me. I often leav…

Hey, Boris from the Claude Code team here. Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache -- everything but the latest message. The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for u…

Prioritize outcomes for users using your product. That should lead to improving the viral/visibility aspect of documentation notification, as well as other aspects of documentation. Make this a differentiator of your product. Widespread misperceptions hurt outcomes.

Could you create one location educating advanced users, and:

• Promote, Organize and Maintain it

• Develop a group of users that have early access to "upcoming notifications we're working on"

• Perhaps give a third party specializing in making information visible responsibility for it

• Read comments by users in various places to determine what should be communicated. Just under this comment @dbeardsl begins "I appreciate the reply, but I was never under the impression that ...".

The speed that key users are informed of issues is critical. This is just off the top of my head, a much better plan I'm sure could be created.

Re: An update on recent Claude Code quality reports

#696

Post by jlokier »

prirun wrote: post #677
theshrike79 wrote: post #650

Earlier quoted context omitted.

Think of it like this: Anthropic has to keep a full virtual machine running just for you. How long should it idle there taking resources when you only pay a static monthly fee and not hourly? They have a limited number of resources and can’t keep everyone’s VM running forever.

I pay $5/mo to Vultr for a VM that runs continuously and maintains 25GB of state.

That price at Vultr gets you 1GB of RAM, and 25GB of relatively slow SSD.

The KV cache of your Claude context is:

- Potentially much larger than 25GB. (The KV cache sizes you see people quoting for local models are for smaller models.)

- While it's being used, it's all in RAM.

- Actually it's held in special high-performance GPU RAM, precision-bonded directly to the silicon of ludicrously expensive, state of the art GPUs.

- The KV state memory has to be many thousands of times faster than your 25GB state.

- It's much more expensive per GB than the CPU memory used by a VM. And that in turn is much more expensive than the SSD storage of your 25GB.

- Because Claude is used by far more people (and their agents) than rent VMs, far more people are competing to use that expensive memory at the same time

There is a lot going on to move KV cache state between GPU memory and dedicated, cheaper storage, on demand as different users need different state. But the KV cache data is so large, and used in its entirety when the context is active, that moving it around is expensive too.

Re: An update on recent Claude Code quality reports

#697

Post by jorblumesea »

FartyMcFarter wrote: post #665
rekrsiv wrote: post #661

Earlier quoted context omitted.

Time is finite and regression testing always gets punted to the back of the line when humans are excited. This simply reveals a staggering level of humanity.

Software engineering is not a new field. Best practices on testing are mature now, and Anthropic has poached enough engineers from companies with a solid understanding of those practices. Yet, their flagship product got three really bad changes shipped into it and only resolved after more than a month. This raises another question: with all the industry-wide boasting about AI-driven productivity, why does the leading…

models are great but models don't magically fix things. you need to set up systems to handle the output of code, you need to instrument metrics to llm to listen to and flag. experimentation is a huge problem, with the huge output of code, how to you keep your business metrics clean and isolate issues. these are all hard challenges.

in response, most companies are explicitly trading velocity for quality, and finding out that quality is actually important at the end of the day. if you look at the roadmap it's just ship ship ship. eng is being told to 3x their output. quality in the llm coded world is tough and there's not much appetite for it right now.

Re: An update on recent Claude Code quality reports

#698

Post by stickfigure »

tuwtuwtuwtuw wrote: post #684
lanyard-textile wrote: post #681

Earlier quoted context omitted.

Eh :) Let's not forget the humans on the other end of this. One of them was a bug that didn't present itself until after an hour of usage.

Seems like that would be trivial to test?

Most bugs are trivial to test for after you know about them.

Re: An update on recent Claude Code quality reports

#699

Post by jimkleiber »

adam_patarino wrote: post #633
jimkleiber wrote: post #433

Earlier quoted context omitted.

I might be willing to pay more, maybe a lot more, for a higher subscription than claude max 20x, but the only thing higher is pay per token and i really dont like products that make me have to be that minutely aware of my usage, especially when it has unpredictability to it. I think there's a reason most telecoms went away from per minute or especially per MB charging. Even per GB, as they often now offer X GB, and i…

Token anxiety is real mental overhead.

That's the phrase i was looking for, thank you.

Re: An update on recent Claude Code quality reports

#700

Post by r00t- »

bcherny wrote: post #124
6keZbCECT2uB wrote: post #61

"On March 26, we shipped a change to clear Claude's older thinking from sessions that had been idle for over an hour, to reduce latency when users resumed those sessions. A bug caused this to keep happening every turn for the rest of the session instead of just once, which made Claude seem forgetful and repetitive. We fixed it on April 10. This affected Sonnet 4.6 and Opus 4.6" This makes no sense to me. I often leav…

Hey, Boris from the Claude Code team here. Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache -- everything but the latest message. The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for u…

We hit limits, and we come back when the limit is lifted. Isn't it obvious sessions are going to stay idle for more than 1 hour when Claude itself is hitting the limits?

I switched to Codex, Claude has gotten to a point where it's just unusable for the regular Joe.

Post reply on HN