Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A postmortem of three recent issues

anthropic.com

Post reply on HN
Print view View original

51–60 of 120 posts

Re: A postmortem of three recent issues

#51

Post by stephen_cagle »

I do wonder what a random dip in quality causes in a long running conversation? Does the conversation recover at a later point, or does the introduction of temporary idiocy permanently affect the rest of the conversation?

Statistically, probably likely that the dip occurred at a point that wasn't too important? But what happens if the idiot comes out at a critical point?

Kind of reminds me of the two alternate ways that time travel works in sci-fi. Does the small change to the past explode like a fission reaction, or does history heal itself?

Anywho, if errors do accumulate, I can see being very pissed off even with temporary idiocy from the model, as it means it poisons the context for the entire rest of the conversation.

Re: A postmortem of three recent issues

#52

Post by willsmith72 »

data-ottawa wrote: post #43

With all due respect to the Anthropic team, I think the Claude status page[1] warrants an internal code red for quality. There were 50 incidents in July, 40 incidents in August, and 21 so far in September. I have worked in places where we started approaching half these numbers and they always resulted in a hard pivot to focusing on uptime and quality. Despite this I'm still a paying customer because Claude is a fanta…

> Despite this I'm still a paying customer because Claude is a fantastic product and I get a lot of value from it.

Doesn't that say it all? At this point the quality of the AI trumps reliability for the customer (you and me), so even though of course they should (and I'm sure will) focus on it, why would they prioritise reliability over model quality right now?

Re: A postmortem of three recent issues

#53

Post by jldugger »

Wowfunhappy wrote: post #13

> On August 25, we deployed a misconfiguration to the Claude API TPU servers that caused an error during token generation. An issue caused by a runtime performance optimization occasionally assigned a high probability to tokens that should rarely be produced given the context, for example producing Thai or Chinese characters in response to English prompts, or producing obvious syntax errors in code. A small subset of…

The AI kernels are floating point, so it's possible to do some unintuitive math that ends up negative even though it wouldn't be in the Real domain. I wouldn't be surprised if checking for overflow state is disabled for perf reasons and the negative simply becomes really big -- like asking for the -1st item in an array and getting the last.

Re: A postmortem of three recent issues

#54

Post by dantodor »

That is a very good start in sharing some level of information with their users, and kudos to the Anthropic team for doing that. However, I don't see any mention of the longstanding issue in CC of API timeout errors. And, at least for me, it's the most frustrating one.

Re: A postmortem of three recent issues

#55

Post by lukasb »

dantodor wrote: post #54

That is a very good start in sharing some level of information with their users, and kudos to the Anthropic team for doing that. However, I don't see any mention of the longstanding issue in CC of API timeout errors. And, at least for me, it's the most frustrating one.

I almost never see these. Maybe issue is your network?

Re: A postmortem of three recent issues

#57

Post by martinald »

lumost wrote: post #49
data-ottawa wrote: post #43

With all due respect to the Anthropic team, I think the Claude status page[1] warrants an internal code red for quality. There were 50 incidents in July, 40 incidents in August, and 21 so far in September. I have worked in places where we started approaching half these numbers and they always resulted in a hard pivot to focusing on uptime and quality. Despite this I'm still a paying customer because Claude is a fanta…

I've become extremely nervous about these sudden declines in quality. Thankfully I don't have a production product using AI (yet), but in my own development experience - the model becoming dramatically dumber suddenly is very difficult to work around. At this point, I'd be surprised if the different vendors on openrouter weren't abusing their trust by silently dropping context/changing quantization levels/reducing ex…

Openrouter is aware this is happening and flags it now on the UI. It's a real problem.

Re: A postmortem of three recent issues

#59

Post by edoceo »

willsmith72 wrote: post #52
data-ottawa wrote: post #43

With all due respect to the Anthropic team, I think the Claude status page[1] warrants an internal code red for quality. There were 50 incidents in July, 40 incidents in August, and 21 so far in September. I have worked in places where we started approaching half these numbers and they always resulted in a hard pivot to focusing on uptime and quality. Despite this I'm still a paying customer because Claude is a fanta…

> Despite this I'm still a paying customer because Claude is a fantastic product and I get a lot of value from it. Doesn't that say it all? At this point the quality of the AI trumps reliability for the customer (you and me), so even though of course they should (and I'm sure will) focus on it, why would they prioritise reliability over model quality right now?

The up-theead complaint is that quality drops and draws a line to reliability. They (Anthropx) have two hard problems to solve.

Re: A postmortem of three recent issues

#60

Post by yomismoaqui »

This reminds me of the story [1] about Facebook intentionally breaking parts of its Android app for some users (including crashing or disabling functionality), to see how far it could degrade before users stopped using Facebook.

According to reports, users did not stop coming back even when the app was broken for hours.

A similar thing happened to me when playing some initial version of The Binding of Isaac on Linux, when it was made with Flash. Its performance wasn't the best but I couldn't stop playing.

So if people still returns maybe Anthropic has something great going on with Claude Code.

[1]: https://www.theguardian.com/technology/2016/jan/05/facebook-...

Post reply on HN