Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude: Elevated Error Rates for Opus 4.8, Opus 4.7, Opus 4.6, and Sonnet 4.6

status.claude.com

Post reply on HN
Print view View original

21–30 of 42 posts

Re: Claude: Elevated Error Rates for Opus 4.8, Opus 4.7, Opus 4.6, and Sonnet 4.6

#24

Post by bombcar »

saghm wrote: post #18
sb057 wrote: post #14

Per their status page, the main product now has one 9 of uptime.

Hopefully still in the "tens" digit!

Good news! We have 6 9s of reliability!

Bad news! You need 128 bit floating point to see them.

Re: Claude: Elevated Error Rates for Opus 4.8, Opus 4.7, Opus 4.6, and Sonnet 4.6

#27

Post by Der_Einzige »

behnamoh wrote: post #7

Times like this remind me that despite GLM and Codex and other models being hyped up as Claude Opus 4.8 replacements, I still would not trust them with my most important work. For example, right now I'm working on a huge refactoring project, and even Opus has struggled with it after several days. I cannot even imagine how GLM, Codex, or other models would handle this. So the only option for me is to wait until this o…

A lot of the perception of open source models being garbage is that they're still using the same piss-poor sampling algorithms that OpenAI/Anthropic force on their users, i.e. Top-p, top-k.

These lead to small accumulation of sampling errors which makes it all but inevitable that open source models will shit the bed by the 200K token mark or even sooner.

If you set your opencode to use a good sampling algorithm, such as min_p or top-n sigma (llamacpp supports both), you'll find that at least for long running tasks, your model gets a lot better.

It won't make GLM as good as Opus 4.8, but it will stop the feeling of "brain damage" from running open source models at the edge of their context windows.

And yes, there is an upcoming (hopefully NeurIPS) paper titled "Long Context Generation is a Sampling Problem" for more details about this. Give it two months and it'll be on Arxiv one way or another.

Re: Claude: Elevated Error Rates for Opus 4.8, Opus 4.7, Opus 4.6, and Sonnet 4.6

#28

Post by jmward01 »

Lots of downtime on CC the last few days. They also pushed an bad release to CC that kept doing 'No response from API · Retrying in .... check your network'. 'claude install stable' fixed that. How I was not on stable I have no idea. I can only guess how many tokens I was billed for sending requests that CC got 'No response from API' for. I have a feeling the big AI vendors won't have the loyalty that IDEs and other dev tools generate. They really need to work on trust now to avoid people hopping later.
Post reply on HN