201–210 of 778 posts
Earlier quoted context omitted.
so who do you trust and go to? (NotClearlySo)OpenAI?
At the moment, yeah. If Google ever figures out how to build an agentic model, I would use them as well. However you feel about OpenAI, at least their harness is actually open source and they don’t send lawyers after oss projects like opencode
Earlier quoted context omitted.
Hey, Boris from the Claude Code team here. Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache -- everything but the latest message. The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for u…
Is having massive sessions which sit idle for hours (or days) at a time considered unusual? That's a really, really common scenario for me. Two questions if you see this: 1) if this isn't best practice, what is the best way to preserve highly specific contexts? 2) does this issue just affect idle sessions or would the cache miss also apply to /resume ?
Edit: If you message me I can share some of my toolchain, it's probably similar to what a lot of other people here use but I've done some polishing recently.
Earlier quoted context omitted.
Why ban third party wrappers? All of this could've been sidestepped had you not banned them.
Because then they lose vertical integration and the extra ability it grants to tune settings to reduce costs / token use / response time for subscription users. Or improve performance and efficiency, if we’re generous and give them the benefit of the doubt. It makes sense, in a way. It means the subscription deal is something along the lines of fixed / predictable price in exchange for Anthropic controlling usage pat…
If you worry about "degraded" experience, then let people choose. People won't be using other wrappers if they turn out to be bad. People ain't stupid.
Earlier quoted context omitted.
I think it’s crazy that they do this, especially without any notice. I would not have renewed my subscription if I knew that they started doing this. Especially in the analysis part of my work I don‘t care about the actual text output itself most of the time but try to make the model „understand“ the topic. In the first phase the actual text output itself is worthless it just serves as an indicator that the context w…
So to defend a litte, its a Cache, it has to go somewhere, its a save state of the model's inner workings at the time of the last message. so if it expires, it has to process the whole thing again. most people don't understand that every message the ENTIRE history of the conversion is processed again and again without that cache. That conversion might of hit several gigs worth of model weights and are you expecting t…
Earlier quoted context omitted.
I see that with openai too, lots of responding to itself. Seems like a convenient way for them to churn tokens.
None of these companies have compute to spare. It’s not in their interest to use more tokens that necessary.
Earlier quoted context omitted.
Hey, Boris from the Claude Code team here. Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache -- everything but the latest message. The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for u…
Is there a way to say: I am happy to pay a premium (in tokens or extra usage) to make sure that my resumed 1h+ session has all the old thinking? I understand you wouldn't want this to be the default, particularly for people who have one giant running session for many topics - and I can only imagine the load involved in full cache misses at scale. But there are other use cases where this thinking is critical - for ins…
The only issue is that it didn't hit the cache so it was expensive if you resume later.
Earlier quoted context omitted.
Is there a way to say: I am happy to pay a premium (in tokens or extra usage) to make sure that my resumed 1h+ session has all the old thinking? I understand you wouldn't want this to be the default, particularly for people who have one giant running session for many topics - and I can only imagine the load involved in full cache misses at scale. But there are other use cases where this thinking is critical - for ins…
I think it’s crazy that they do this, especially without any notice. I would not have renewed my subscription if I knew that they started doing this. Especially in the analysis part of my work I don‘t care about the actual text output itself most of the time but try to make the model „understand“ the topic. In the first phase the actual text output itself is worthless it just serves as an indicator that the context w…
1. Specifically, this suite was about price increases without clear consideration for both parties - but the same justifications apply to service restrictions without corresponding price decreases.
https://fortune.com/2026/04/20/italian-court-netflix-refunds...
Earlier quoted context omitted.
> My hypothesis is that some of this a perceived quality drop due to "luck of the draw" where it comes to the non-deterministic nature of [LLM] output. I think you must have learned that they’re more nondeterministic than you had thought, but then wrongly connected your new understanding to the recent model degradation. Note: they’ve been nondeterministic the whole time, while the widely-reported degradation is recen…
Er, no, I am fully aware that LLMs have always been non-deterministic.
I see some anthropic claude code people are reading the comments. A day or two ago I watched a video by theo t3.gg on whether claude got dumber. Even though he was really harsh on anthropic and said some mean stuff. I thought some of the points he was raising about claude code was quite apt. Especially when it comes to the harness bloat. I really hope the new features now stop and there is a real hard push for polish…