Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An update on recent Claude Code quality reports

anthropic.com

Post reply on HN
Print view View original

221–230 of 778 posts

Re: An update on recent Claude Code quality reports

#221

Post by jetbalsa »

dicethrowaway1 wrote: post #211
jetbalsa wrote: post #194

Earlier quoted context omitted.

The cache is on Antropics server, its like a freeze frame of the LLM inner workings at the time. the LLM can pick up directly from this save state. as you can guess this save state has bits of the underlying model, their secret sauce. so it cannot be saved locally...

Maybe they could let users store an encrypted copy of the cache? Since the users wouldn't have Anthropic's keys, it wouldn't leak any information about the model (beyond perhaps its number of parameters judging by the size).

I'm unsure of the sizes needed for prompt cache, but I suspect its several gigs in size (A percentage of the model weight size), how would the user upload this every time they started a resumed a old idle session, also are they going to save /every/ session you do this with?

Re: An update on recent Claude Code quality reports

#222

Post by eknkc »

3836293648 wrote: post #205
jetbalsa wrote: post #197

Earlier quoted context omitted.

So to defend a litte, its a Cache, it has to go somewhere, its a save state of the model's inner workings at the time of the last message. so if it expires, it has to process the whole thing again. most people don't understand that every message the ENTIRE history of the conversion is processed again and again without that cache. That conversion might of hit several gigs worth of model weights and are you expecting t…

No? It's not because it's a cache, it's because they're scared of letting you see the thinking trace. If you got the trace you could just send it back in full when it got evicted from the cache. This is how open weight models work.

I’m not familiar with the Claude API but OpenAI has an encrypted thking messages option. You get something that you can send back but it is encrypted. Not available on Anthropic?

Re: An update on recent Claude Code quality reports

#223

Post by tbrockman »

elAhmo wrote: post #207
btown wrote: post #169

Earlier quoted context omitted.

Is there a way to say: I am happy to pay a premium (in tokens or extra usage) to make sure that my resumed 1h+ session has all the old thinking? I understand you wouldn't want this to be the default, particularly for people who have one giant running session for many topics - and I can only imagine the load involved in full cache misses at scale. But there are other use cases where this thinking is critical - for ins…

Don't you have that by just resuming old convo? The only issue is that it didn't hit the cache so it was expensive if you resume later.

Or generate tiny filler messages every hour until you come back to it.

Re: An update on recent Claude Code quality reports

#224

Post by deckar01 »

y1n0 wrote: post #65
dawnerd wrote: post #60

Earlier quoted context omitted.

I see that with openai too, lots of responding to itself. Seems like a convenient way for them to churn tokens.

None of these companies have compute to spare. It’s not in their interest to use more tokens that necessary.

You don’t have to use compute to pad the token count.

Re: An update on recent Claude Code quality reports

#225

Post by throwaway2027 »

rishabhaiover wrote: post #79

Boris gaslighted us with all the quality related incidents for weeks not acknowledging these problems.

Maybe he didn't know or they were still figuring it out which is fine they're still engineers who can get things wrong sometimes but the communication felt lackluster and being on the receiving end sucks when you had a reliable setup which then degrades. There is a reason people don't upgrade software and why people say if it works don't fix it, but obviously that's not an option for Anthropic when you want to keep improving the product, so they need good measurement tools and quick rollbacks even if properly "benchmarking" LLMs could prove difficult.

Re: An update on recent Claude Code quality reports

#226

Post by cmrdporcupine »

carlgreene wrote: post #73
mannanj wrote: post #15

Earlier quoted context omitted.

so who do you trust and go to? (NotClearlySo)OpenAI?

I "subconsciously" moved to codex back in mid Feb from CC and it's been so freaking awesome. I don't think it's as good at UI, but man is it thorough and able to gather the right context to find solutions. I use "subconsciously" in quotes because I don't remember exactly why I did it, but it aligns with the degradation of their service so it feels like that probably has something to do with it even though I didn't re…

Codex isn't great at UI, but you might find Gemini is competent enough as an adjunct. I've had some luck with that.

Re: An update on recent Claude Code quality reports

#227

Post by cmrdporcupine »

IncreasePosts wrote: post #201
Robdel12 wrote: post #46

Earlier quoted context omitted.

At the moment, yeah. If Google ever figures out how to build an agentic model, I would use them as well. However you feel about OpenAI, at least their harness is actually open source and they don’t send lawyers after oss projects like opencode

Is Gemini cli not an agentic model? Or are you just saying it's built poorly? Gemini 2.5 didn't really work for me but Gemini 3 seems fairly solid

Gemini fairs poorly at tool use, even in its own CLI and even in Antigravity. It gets into a mess just editing source files, it's tragic because it's actually not a bad model otherwise.

Re: An update on recent Claude Code quality reports

#228

Post by robeym »

podnami wrote: post #77

They lost me at Opus 4.7 Anecdotally OpenAI is trying to get into our enterprise tooth and nail, and have offered unlimited tokens until summer. Gave GPT5.4 a try because of this and honestly I don’t know if we are getting some extra treatment, but running it at extra high effort the last 30 days I’ve barely see it make any mistakes. At some points even the reasoning traces brought a smile to my face as it preemptive…

What's your workflow like? I'd be curious to test OpenAI out again but Claude Code is how I use the models. Does it require relearning another workflow?

Re: An update on recent Claude Code quality reports

#230

Post by [deleted] »

jetbalsa wrote: post #197
CjHuber wrote: post #187

Earlier quoted context omitted.

I think it’s crazy that they do this, especially without any notice. I would not have renewed my subscription if I knew that they started doing this. Especially in the analysis part of my work I don‘t care about the actual text output itself most of the time but try to make the model „understand“ the topic. In the first phase the actual text output itself is worthless it just serves as an indicator that the context w…

So to defend a litte, its a Cache, it has to go somewhere, its a save state of the model's inner workings at the time of the last message. so if it expires, it has to process the whole thing again. most people don't understand that every message the ENTIRE history of the conversion is processed again and again without that cache. That conversion might of hit several gigs worth of model weights and are you expecting t…

[deleted]
Post reply on HN