Reminder that Anthropic is the only AI company that has never released any open-source/weight models.
Sure, but don't you feel safer that way?
A postmortem of three recent issues
31–40 of 120 posts
Re: A postmortem of three recent issues
#32> On August 29, a routine load balancing change unintentionally increased the number of short-context requests routed to the 1M context servers. At the worst impacted hour on August 31, 16% of Sonnet 4 requests were affected. Interesting, this implies that the 1M context servers performs worst at low context. Perhaps this is due to some KV cache compression, eviction or sparse attention scheme being applied on these…
[dead]
Re: A postmortem of three recent issues
#33Re: A postmortem of three recent issues
#34If you are going to run a non deterministic system on three very different hardware platforms doesn't it behoove you to tell your users where their experience is coming from?
Calling the platforms A, B and C might help provide us the insight we're missing to spot incongruous behaviors faster than trying to aggregate more generalized feedback.
Re: A postmortem of three recent issues
#35Earlier quoted context omitted.
Is your contention that paying for a service entitles you to zero bugs, ever?
Of course not! But usually, you can quantify metrics for quality, like uptime, lost transactions, response time, throughput etc. Then you can have accountability, and remediate. Even for other bugs, you can often reproduce and show clearly the impact. But in this case, other than internal benchmarks, you cannot really prove it. There is no accountability yet
why would they publish the data you seek? I would not publish it either.
the blog explains what issues they had and how they fixed them. this is good enough.
Re: A postmortem of three recent issues
#36> On August 29, a routine load balancing change unintentionally increased the number of short-context requests routed to the 1M context servers. At the worst impacted hour on August 31, 16% of Sonnet 4 requests were affected. Interesting, this implies that the 1M context servers performs worst at low context. Perhaps this is due to some KV cache compression, eviction or sparse attention scheme being applied on these…
This is due to RoPE scaling.
> All the notable open-source frameworks implement static YaRN, which means the scaling factor remains constant regardless of input length, potentially impacting performance on shorter texts. We advise adding the rope_scaling configuration only when processing long contexts is required. It is also recommended to modify the factor as needed. For example, if the typical context length for your application is 524,288 tokens, it would be better to set factor as 2.0.
Re: A postmortem of three recent issues
#37I’m pretty surprised that Anthropic can directly impact the infra for AWS Bedrock as this article suggests. That goes against AWSs commitments. I’m sure the same is true for Google Vertex but I haven’t digged in there from a compliance perspective before.
> Our own privacy practices also created challenges in investigating reports. Our internal privacy and security controls limit how and when engineers can access user interactions with Claude, in particular when those interactions are not reported to us as feedback.
Ok makes sense and glad to hear
> It remains particularly helpful for users to continue to send us their feedback directly. You can use the /bug command in Claude Code
Ok makes sense and I’d expect that a human can then see the context in that case although I hope it is still very explicit to the end user (I’m not a Claude Code user so I cannot comment)
> or you can use the "thumbs down" button in the Claude apps to do so
This is pretty concerning. I can’t imagine the average person equates hitting this button with forfeiting their privacy.
Re: A postmortem of three recent issues
#38I’m pretty surprised that Anthropic can directly impact the infra for AWS Bedrock as this article suggests. That goes against AWSs commitments. I’m sure the same is true for Google Vertex but I haven’t digged in there from a compliance perspective before. > Our own privacy practices also created challenges in investigating reports. Our internal privacy and security controls limit how and when engineers can access use…
> This is pretty concerning. I can’t imagine the average person equates hitting this button with forfeiting their privacy.
When you click "thumbs down" you get the message "Submitting this report will send the entire current conversation to Anthropic for future improvements to our models." before you submit the report, I'd consider that pretty explicit.
Re: A postmortem of three recent issues
#39The value of figuring out how to make their LLM serving deterministic might help them track this down. There was a recent paper about how the received wisdom that kept assigning it to floating point associativity actually overlooked the real reasons for non-determinism [1].
[1] https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
Re: A postmortem of three recent issues
#40I’m pretty surprised that Anthropic can directly impact the infra for AWS Bedrock as this article suggests. That goes against AWSs commitments. I’m sure the same is true for Google Vertex but I haven’t digged in there from a compliance perspective before. > Our own privacy practices also created challenges in investigating reports. Our internal privacy and security controls limit how and when engineers can access use…
(Anthropic employee, speaking in a personal capacity)
> I’m pretty surprised that Anthropic can directly impact the infra for AWS Bedrock as this article suggests.
We don't directly manage AWS Bedrock deployments today, those are managed by AWS.
> I can’t imagine the average person equates hitting this button with forfeiting their privacy.
We specify
> Submitting this report will send the entire current conversation to Anthropic for future improvements to our models.
in the thumbs down modal. Is there a straightforward way to improve this copy?