[flagged]
If the government wanted people to take holidays off they could just legislate that people can't work on those days. I doubt there is any political will to do this, though.
Elevated error rate across multiple models
231–240 of 293 posts
Re: Elevated error rate across multiple models
#232Earlier quoted context omitted.
I think we need to disqualify humans as well. Their brains have been shown to operate on probabilistic chemical interactions and even quantum effects.
That doesn’t disqualify humans. It highlights the difference I am talking about. Those chemical interactions and quantum effects lead to emergent properties like judgment, experience, context, accountability, and an understanding of consequences. Those are not properties that LLMs possess, regardless of how useful their output can be. That is not to say that, in the future, LLMs won’t be used as part of other systems…
My understanding is that the quantum effects has 0 impact, see https://en.wikipedia.org/wiki/Orchestrated_objective_reducti.... It's currently fringe/unproven science.
Re: Elevated error rate across multiple models
#233Earlier quoted context omitted.
This is almost certainly a software issue, though. Even if it's due to scaling, they still built a system that failed catastrophically rather than degrading gracefully.
Right. If this were truly a pure scaling issue, I’d expect the interface would offer an archive.is-esque “Claude is at capacity; your prompt is #XXX/YYY in the queue; estimated time remaining: ZZZ seconds” Instead, the whole system just shits the bed, catastrophically.
Except these models are not run prompt-to-prompt. The infra has to hold the entire context.
Re: Elevated error rate across multiple models
#234Earlier quoted context omitted.
Is there any indication these errors are related to Anthropic-written code as opposed to operational issues from the fastest-growing infra buildout ever? Layer-wise, the app is pretty far removed from request routing to GPU pools.
This is almost certainly a software issue, though. Even if it's due to scaling, they still built a system that failed catastrophically rather than degrading gracefully.
> failed catastrophically rather than degrading gracefully
You mean like returning 529s and operating with reduced QoS?
Re: Elevated error rate across multiple models
#235I have been developing software since the late 80s, mostly CAM software for metal cutting machines, and I have been refereeing tabletop roleplaying games like Dungeons & Dragons since the late 70s. I get the power of LLMs, and I do find them useful. But I find them useful in much the same way I find a really good set of random tables useful, or a good set of rules for procedurally generating something like a star sec…
> I am failing to see how the inherent probabilistic nature of the technology can be fully overcome.
This is common in image generation pipelines because if you find an image you really like, you can store the seed and then reproduce it with small tweaks, otherwise - to quote Borges - “Look at it well. You will never see it again" User-facing deterministic pipelines do exist for generative AI.
I know you make this argument in your post, but that's really the answer if you want repeatable results. For a classifier or a detector, determinism is a requirement, but for an LLM non-determinism desirable property because it feels like a more natural conversation. The downside is it's extremely difficult to replicate a response without pointing the model to an earlier conversation.
And specifically for the RPG case, don't you want non-determinism? You don't want the model spinning up the same identical person if you say "Generate me an NPC character sheet for an innkeeper". This was a complaint that people had in the past, that models would regurgitate the same scenarios or the same jokes.
Where I suspect DMs run into trouble is not randomness, but lack of self-consistency in worldbuilding. Say you generate an NPC and then refer back to them later and the model gets some details wrong. You could compare to a system like Dwarf Fortress where everything down to the genealogy and faction relationships are rigidly generated.
Re: Elevated error rate across multiple models
#236The rainbow has to keep being a rainbow. ClaudeCode still has a 99.27 % uptime ClaudeCowork has 99.52 % uptime ClaudeForGovernment has 99.93 % uptime
I must be unlucky because I'm in that .73% way more than .73% of the time.
There was a post on here the other day explaining this exact phenomenon: https://brooker.co.za/blog/2026/06/19/waiting.html
Re: Elevated error rate across multiple models
#237Earlier quoted context omitted.
I keep butting into the question of; why opencode, when you've got codex available? Codex is open source as well, and i can't seem to picture a situation where one would want Opencode over Codex. As far as I can tell, they tick the same boxes- but one has the support of a big boy model provider.
Well, the reason is simple: over the past several months, it has become very difficult to use Codex with non-OpenAI models. They removed the old edit tool that didn't require OpenAI's free form tool calling (that no other LLM host supports), they are adding tools to every request of a type that break most LLM hosts unless you use a proxy to filter them out, they add a "developer" role to some messages which breaks so…
Ah, I wasn't aware things regressed there. Yea certainly workarounds n soft fork sorts of things definitely would work- but thats a bummer than things have changed.
From watching Pr's and issues- seems like openai at least wants to come across as if theyre supporting non-oai models :/
Re: Elevated error rate across multiple models
#238Earlier quoted context omitted.
Well, the reason is simple: over the past several months, it has become very difficult to use Codex with non-OpenAI models. They removed the old edit tool that didn't require OpenAI's free form tool calling (that no other LLM host supports), they are adding tools to every request of a type that break most LLM hosts unless you use a proxy to filter them out, they add a "developer" role to some messages which breaks so…
Ah, I wasn't aware things regressed there. Yea certainly workarounds n soft fork sorts of things definitely would work- but thats a bummer than things have changed. From watching Pr's and issues- seems like openai at least wants to come across as if theyre supporting non-oai models :/
Yeah... one of the relevant issues: https://github.com/openai/codex/issues/11940#issuecomment-45...
You would think they would support their own GPT-OSS model, but, not really anymore. I wish they would release a GPT-OSS 2, but this doesn't fill me with confidence.
Re: Elevated error rate across multiple models
#239I have been developing software since the late 80s, mostly CAM software for metal cutting machines, and I have been refereeing tabletop roleplaying games like Dungeons & Dragons since the late 70s. I get the power of LLMs, and I do find them useful. But I find them useful in much the same way I find a really good set of random tables useful, or a good set of rules for procedurally generating something like a star sec…
Setting aside that we're living in a universe that's full of (practically) deterministic processes built over probabilistic components (and which behave sufficiently reliably without any human in the loop), I think the specific failure mode you're citing is that there aren't enough gates and constraints applied to the processes you've seen.
LLMs can contribute quite reliably given very narrow prompts and short horizons (keeping turns low and context brief). If you chain a bunch of these narrow contributions together and define guardrails (structured outputs, online evals, other-llm-as-judge/jury, etc...) you can produce a very repeatable workflow that reliably delivers to defined service levels.
The obvious issue being - you've got to define the workflow and implement all the guardrails, not hope that the LLM will infer them during a session or a one-shot prompt.
Re: Elevated error rate across multiple models
#240I have been developing software since the late 80s, mostly CAM software for metal cutting machines, and I have been refereeing tabletop roleplaying games like Dungeons & Dragons since the late 70s. I get the power of LLMs, and I do find them useful. But I find them useful in much the same way I find a really good set of random tables useful, or a good set of rules for procedurally generating something like a star sec…
The trouble is: there is no deterministic algorithm that can do the things neural networks can do.
For many of these problems, I think it is likely that no deterministic algorithm can exist because the problems are fundamentally underspecified. E.g. a common task in computer vision is generating a 3D depth map from a 2D image. This is inverting a lossy projection, so any solution must be a least partially a hallucination.
I think we just have to accept this. It's a different type of algorithm, built out of statistics instead of logic, with different strengths and weaknesses compared to traditional software.