Live data from Hacker News

Elevated error rate across multiple models

status.claude.com

191–200 of 293 posts

Re: Elevated error rate across multiple models

#191
post #59
post #38

Earlier quoted context omitted.

"curl -fsSL https://pi.dev/install.sh | sh" — seriously? That tells me a lot about the whole project, unfortunately.

I am genuinely curious what it tells you, as "curl https//.. | sh" has long been an enormously popular approach to distribution in the open source world. Homebrew, to name just one example, advertises a similar method. (pi.sh also documents other install methods, like `npm`, on their homepage) If trust and security is the issue, unfortunately "better" ideas like hashpipe [1] never achieved critical mass [1] https://n…

> I am genuinely curious what it tells you, as "curl https//.. | sh" has long been an enormously popular approach to distribution in the open source world.

It's plain horrible. You could have, for example, a compromised server serving malware but only one out of every 100 download. The only signature you rely on is TLS.

Proper package distribution are using proper signatures schemes, are decentralized, even for some offer reproducible builds (meaning you can rebuild the whole package yourself and verify your build matches), etc.

Hashpipe is an attempt at reproducing some of those guarantees. Not unlike container pining using hashes. It at least fixes the "Jack and John installed this already and I know I'm getting the same version as they did".

Proper software distribution is signed, reproducible and ideally also uses some proof-of-existence for the hashes.

My bet is this: in the face of the countless supply chain attacks, we'll see more and more people getting very serious about security, including the security of software distribution. And curl bash'ing won't be part of it.

Re: Elevated error rate across multiple models

#192

Earlier quoted context omitted.

This is almost certainly a software issue, though. Even if it's due to scaling, they still built a system that failed catastrophically rather than degrading gracefully.

Right. If this were truly a pure scaling issue, I’d expect the interface would offer an archive.is-esque “Claude is at capacity; your prompt is #XXX/YYY in the queue; estimated time remaining: ZZZ seconds” Instead, the whole system just shits the bed, catastrophically.

But such messages would suggest that Claude has engineered limits, which isn't what the market wants to hear. Completely falling over and being unavailable is just another Tuesday on the internet, will be forgotten by the weekend.

Re: Elevated error rate across multiple models

#193
post #163

Earlier quoted context omitted.

Engineering practices or best practices are much more than writing code. So not sure what we are debating here: I see first hand companies jumping full on using LLM for _everything_ for the last 6 months (of course Anthropic longer) and without guardrails and good engineering practices the number of incidents, downtime is increasing. Look at status.claude.com - Anthropic could at any point come out and say all those…

> Anthropic could at any point come out and say all those are due to third party providers. Why can't it be simply the case that Anthropic is struggling by their own accord? Infra scaling isn't a solved problem, much less with new, complicated, ever-changing, stateful LLM requests. Pretty much every API-service-centric company I've worked at was in some constant state of either triaging or thinking about infrastructu…

> Why can't it be simply the case that Anthropic is struggling by their own accord?

They can and it is normal. I have said it is normal for scaleups specifically at a similarity growth rate.

What we (or at least I) critique here is coming out in the world and announcing that coding is done while having a product that has a status page full with red stripes. Yes, could be infrastructure, could be third party integrations could be a lot. But a lot of what is there is software. And yes, some parts is hardware. Unless the root cause is culture. In that case as I mentioned in another comment there I give them that: LLMs cannot solve culture.

Again the difference here is that the other scale-ups with similar _scaling_ issues are not talking about how we should all just use LLMs for everything and that learning to code is not required anymore.

So I am not saying the real issue is not infra or integration with third parties. What I am pointing at is: "don't talk that you don't need engineering while you - yourself - have engineering problems that need engineering solutions and still have not solve them".

Also you are getting out of your way to brainstorm possible root causes that will let them get away with this cognitive dissonance (or is there a better them in communication). Let them do the explanation and defend their position as they are the ones attacking the computer science engineering.

Re: Elevated error rate across multiple models

#194
post #172

Earlier quoted context omitted.

You just said "it's not the 'must be veganism' thing, it's the 'must be veganism thing'" Unless you have inside knowledge of their infra ops and management tools, it is just guessing and blaming veganism. For all we know it could be tools from Nvidia or anyone else failing under massive load. It could be the veganism. Some things are. Leaping to it as the only possible explanation for every ailment is exactly the fal…

No. We dont need metaphors like that with veganism (which touches ideologies also) when talking about engineering and a company that promotes out loud that engineering is done. I have not stated anything. I just replied to a metaphor which is not needed cause here we talk about engineering problems handled by engineers in a tech company. I give you something else where this line of thought could be wrong: culture bea…

If you're really committed to the "no difference between datacenter hardware engineering and claude code harness engineering, they must all use the same practices, anything true for one is true for the other" bit, fair enough. It seems fairly ideological to me.

Re: Elevated error rate across multiple models

#195

Earlier quoted context omitted.

Is there any indication these errors are related to Anthropic-written code as opposed to operational issues from the fastest-growing infra buildout ever? Layer-wise, the app is pretty far removed from request routing to GPU pools.

This is almost certainly a software issue, though. Even if it's due to scaling, they still built a system that failed catastrophically rather than degrading gracefully.

Sure. But could it be k8s config? Could it be Nvidia Bright Cluster? Could it be load balancing?

I'm not saying Anthropic isn't to blame for a system that is literally approaching one-nine uptime; they certainly are. I am saying that jumping to the "it must be vibe coding's fault" is an emotional confirmation-bias belief, not an evidence-based belief.

Re: Elevated error rate across multiple models

#196
post #46

Earlier quoted context omitted.

Claude is 0.89% downtime. Getting close to one nine. There aren't many tools that remain useful at that rate.

That's two nines. One nine would be 10% downtime.

So 95% uptime / 5% downtime is two nines?

Re: Elevated error rate across multiple models

#197
post #172

Earlier quoted context omitted.

No. We dont need metaphors like that with veganism (which touches ideologies also) when talking about engineering and a company that promotes out loud that engineering is done. I have not stated anything. I just replied to a metaphor which is not needed cause here we talk about engineering problems handled by engineers in a tech company. I give you something else where this line of thought could be wrong: culture bea…

If you're really committed to the "no difference between datacenter hardware engineering and claude code harness engineering, they must all use the same practices, anything true for one is true for the other" bit, fair enough. It seems fairly ideological to me.

It feels to me that you are really trying hard to brainstorm root causes for their failures.

Could be a hardware issue, could be a datacenter issue. Could be anything. So it could also be a software issue right?

This is not ideology. This is talking about root causes and I replied to someone that started saying this is not because of them promoting and using LLMs to the maximum. Could be it is not because of that. But it could be because of LLMs.

Keeping a company accountable when they try to sell a service that will replace engineers is not ideology. Ideology will be to not use any company that uses LLMs. But pointing out the disconnect between the public discourse and the status.claude.com is a simple idea.

Can you tell me that all those red lines there are infra?

Re: Elevated error rate across multiple models

#198
post #187

I don’t prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops. — Boris Cherny, head of Claude Code Reliability is a direct reflection of the quality of the underlying infrastructural code. If even Anthropic, the company with the world's best agentic vibecoders, has horribly unreliable infrastructure, it really says something about the quality of the world…

He is a salesman at this point and is not talking to you. He is talking to the investors who want to vibe code loops to waste tokens on building slop to get rid of you. Goes to show how fake this industry has become when VC dollars have flooded it. Somehow it is fine to vibe code infrastructure or security because someone (with a clear vested interest) wants you to spend more tokens at their casino because that is ho…

It's very very clear they're eating their own dog food, in a product space built on tech that didn't really exist publicly 5 years ago, to the success of billions, that people increasingly depend on. Maybe I'm an optimist, but I can't fathom the intense negativity or perspective of failure here.

Don't use it. Maybe wait a few more years. If it's not valuable/useful, then not using it, while everything matures, will not be a problem.

Re: Elevated error rate across multiple models

#199

Earlier quoted context omitted.

Meh, this is the "must be the veganism" fallacy: if someone knows you're vegan, then any ailment you might have, no matter how ubiquitous in the population, must be somehow due to your vegan diet and no more details are required. Except now it's the "AI did it" fallacy where if you know a company uses AI, even infra scaling issues must be due to AI, and if you had just used less or no AI, you would have been spared e…

Ahem... "Vegans and vegetarians may have higher stroke risk" - https://www.bbc.com/news/health-49579820 "Vegans had a 43% higher risk of fractures overall compared to nonvegetarians, as well as higher risks of hip, leg, and vertebral fractures." - https://sniglobal.org/plant-based-diets-and-fracture-risk/ "The Impact of a Vegan Diet on Many Aspects of Health: The Overlooked Side of Veganism" - https://www.cureus.com/…

This isn't really the place for this, nor does it matter to my analogy.

But I was more getting at, say, staying out of the sun or being skinnyfat as a vegan, and suddenly you look "sickly"/"frail" when you'd be given the grace of looking like most people otherwise.

A similar analogy would be someone saying "well, of course you do" if you have any malady while having been vaccinated. My point being to bring up the thought terminating cliche of it compared to doing the necessary further analysis to link the malady with the suspected cause.

---

> "Vegans and vegetarians may have higher stroke risk"

It was a lump vegetarian + vegan group with a weak CI bounded at 1.02 for 3/1000 cases over a decade. The same group also had a more robust benefit of less heart disease than meat eaters. The stroke outcomes aren't replicated in other cohorts either, afaik. But the heart disease benefits are.

> "Vegans had a 43% higher risk of fractures overall compared to nonvegetarians, as well as higher risks of hip, leg, and vertebral fractures."

The study used a single baseline questionnaire for 17+ years and looked at vegans with correctable nutrition deficiencies to see +15/1000 hip fractures over 10 years. I'll grant that a poorly planned diet, especially 30 years ago with less nutritional understanding, has worse health outcomes. Just like I wouldn't use the average American's diet to lambast an omnivore diet (compared to, say, the "Mediterranean" diet).

> "vegans had lower iodine, bone health scores" (RBVD study)

On bones: p=0.02 in 72 people with 5% less QUS score in their heel bone (not DXA nor bone density tested). No body weight mediation nor data about health outcomes like fractures, osteoporosis, and no time dimension since it was just a snapshot (cross-sectional).

On iodine: It's a surrogate biomarker from a single pee test. Study didn't look at iodine-related health outcomes like thyroid dysfunction, goiter, or clinical consequences.

---

Re: Elevated error rate across multiple models

#200

Earlier quoted context omitted.

This is almost certainly a software issue, though. Even if it's due to scaling, they still built a system that failed catastrophically rather than degrading gracefully.

Sure. But could it be k8s config? Could it be Nvidia Bright Cluster? Could it be load balancing? I'm not saying Anthropic isn't to blame for a system that is literally approaching one-nine uptime; they certainly are. I am saying that jumping to the "it must be vibe coding's fault" is an emotional confirmation-bias belief, not an evidence-based belief.

I'd expect that they're also managing their k8s config and other infra using LLMs (it's actually quite good at this, at least for my simple homelab use-cases).
Post reply on HN