Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Degraded performance for multiple models

status.claude.com

Post reply on HN
Print view View original

121–130 of 149 posts

Re: Degraded performance for multiple models

#121

Post by bushido »

vidarh wrote: post #114
bushido wrote: post #81

It's very interesting. I think Anthropic's early success in coding/tooling resulted in a lot of workflows using claude. I have started using every bit of my spare capacity to now move off these workflows. It's almost at a point now that if I use anything but Fable, the quality is subpar, Compared to alternatives (closed and open). The only reason I use Fable is because my harnesses still depend on claude code.

You can use Claude Code directly with any provider that supports Anthropic's API by setting some environment variables, and indirectly via a proxy with pretty much anything else.

You can, but claude is better at working with it's own tool calls. Other models work great as a drop-in into omp/opencode etc, but in my experience not as much with CC.

I think there are some anti-patterns in CC that cause the issue - less a deficiency with other models.

Not to mention a lot of the harness is just built around the misbehaviors of anthropics models.

It's a lot of the instruction when it gets given to other models actually degrades their performance, not because the models are bad, but because they don't have the same underlying issues as Claude.

Re: Degraded performance for multiple models

#122

Post by logicchains »

paxys wrote: post #100

Monthly uptime: Claude API - 99.27% Claude Code - 99.16% Claude.ai - 99.14% At any large tech company these numbers would get entire teams of engineers fired. Anthropic, meanwhile, has been busy selling its “better than human engineers” AI while not managing to crack three 9s of availability.

>At any large tech company these numbers would get entire teams of engineers fired

Is Github not a large tech company?

Re: Degraded performance for multiple models

#123

Post by ipsod »

lysace wrote: post #69
echelon wrote: post #34

Earlier quoted context omitted.

LLMs for coding are new. There are lots of alternatives, and there's a burgeoning open source compliment. We'll be fine no matter how Anthropic fares.

Yeah, compared to AWS the lock-in effect is tiny. I'm sure there are highly prioritized plans to "improve" on this. I guess they would need to control/"own" more of their customers data in proprietary formats. Not markdown/source code in English with agents running on customers' machines. Something cloud/web-based, "preferably".

Be nice if you could just "own" their RAM/GPU, wouldn't it?

Re: Degraded performance for multiple models

#124

Post by dofm »

arein3 wrote: post #109
dofm wrote: post #75

Earlier quoted context omitted.

I am fully unbothered about Amodei's wife trying to make high end porn for women, in the same way that I am fully unbothered by Melania Trump having been essentially a glamour/nude model. I know (and have creatively worked) with women who do/have done both; they are better, less hypocritical, more grounded humans than many others. Both Cami Clark and Melania Trump can properly be judged on their involvement with Trum…

"High end" porn business is usually linked with other "services", my guess is that his wife saw value not in revenue from porn but from influence the other services may bring. Of course this is speculative, but this is my guess about her. Darios assiciation with her also indicates a lot about his moral compass. Imagine what we don't know. Also there is a very big difference for a woman to participate in porn (usually…

High end porn can just be expensive production values; that is what I assumed. She wanted a proper studio and production company, and the implication is she was positioning it in opposition to the handful of studios that dominate that industry who have a very male outlook. She did want an e-commerce platform attached, but again, that's commonplace.

None of that is necessarily indicative of trafficking or prostitution, but banks and payment providers tend to run scared of it, so I suppose if one does want serious investment to make a studio, there are relatively few people you can go to.

She went to Epstein because she knew he was very rich and unbothered by that association, I am sure. The fact that she must also have known who he was — a sex offender who basic due diligence would have told her was likely a trafficker — is what is ugly.

What is bizarre about it is that if she had taken his money then needed to go to payment providers, she would have had an extra millstone around her neck, because they definitely do due diligence. But maybe Epstein had clout enough to offset his reputation.

Not going to too get into questions of porn and regret, except to say that even in a post-onlyfans world I think "regret" is often shorthand for "made to regret", which is rather different. It's the same for art and life models, which is where my creative experience lies. People are always out there trying to project their morality onto things that are not their business, so subsequent "regret" is often a practical, imposed matter.

This is why I think it's important that we judge these women for the clearer questions of morality that have nothing to do with porn.

Re: Degraded performance for multiple models

#125

Post by binoct »

danieltk76 wrote: post #119
binoct wrote: post #79

Earlier quoted context omitted.

I hope you don’t plan to cut back on reading legal documents crafted by any LLM before executing them.

I read everything. I will have AI ingest NDAs to make sure they arent glaringly weird and I then go read them, it gives me a good idea of what to look for.

It’s more that unwanted disclosure of sensitive business numbers is a product-class limitation at this time. While undoubtedly some models are better than others, expecting them not to leak information in high stakes output is unseasonable. Seems like you agree, since you are reading everything.

Jumping to another company’s offering because of one instance doesn’t seem like that’s going to meaningfully change your experience. I’m guessing there was more to it, but that’s how it came across in your first post.

Re: Degraded performance for multiple models

#126

Post by Bluestein »

rsoto2 wrote: post #113
chasd00 wrote: post #99

Earlier quoted context omitted.

"Thank God i checked to see if the gun was loaded before i pointed it in a random direction and pulled the trigger". sheesh, my assumptions of general human intelligence continues to be wrong.

the last few years of reading hackernews has been a wild ride. I used to come here for well thought out articles and opinions. IDK if all the "older guard" have left the building or if we've all been consumed by abject stupidity.

Or, both?

Re: Degraded performance for multiple models

#127

Post by deadbunny »

logicchains wrote: post #122
paxys wrote: post #100

Monthly uptime: Claude API - 99.27% Claude Code - 99.16% Claude.ai - 99.14% At any large tech company these numbers would get entire teams of engineers fired. Anthropic, meanwhile, has been busy selling its “better than human engineers” AI while not managing to crack three 9s of availability.

>At any large tech company these numbers would get entire teams of engineers fired Is Github not a large tech company?

No, it is a platform owned and managed by Microsoft.

Well, owned and mismanaged by Microsoft.

Re: Degraded performance for multiple models

#128

Post by rsoto2 »

Bluestein wrote: post #126
rsoto2 wrote: post #113

Earlier quoted context omitted.

the last few years of reading hackernews has been a wild ride. I used to come here for well thought out articles and opinions. IDK if all the "older guard" have left the building or if we've all been consumed by abject stupidity.

Or, both?

probably

Re: Degraded performance for multiple models

#129

Post by __MatrixMan__ »

gaigalas wrote: post #33
__MatrixMan__ wrote: post #23

Earlier quoted context omitted.

Nothing new here. Except for the most trivial of bugs, finding and reliably replicating the bug is almost always harder than fixing it.

I lived in a short period of time in which QA was really good. Early Jenkins era, before GitHub. People engineered a lot of ingenious stuff to prevent bugs. One team I worked with had tests for the product we made ranging from IE6 to IE11, for example. We did demos in-company where people would poke at the products before launch, play with it. When it reached production, it was rock solid stuff. Our motto was "qualit…

Agreed, which is a shame. I'd love to see people with highly developed QA skills using AI to push the envelope. There's so much that's possible now that wasn't 15 years ago.

For instance "formal verification" has been a dirty word, but now that you can write a proof in lean and have an AI generate an implementation which satisfies it, it seems the bounds of what's economical has changed in a very pro-QA direction.

Not to say that that's the silver bullet, but there are many similar examples worth exploring.

But I've been interviewing SDETs lately and maybe I've just been unlucky but I don't see a lot of candidates that are ready to rise to meet this challenge.

We stopped tending to that garden and now that we have a recipe that calls for it's fruits, they're underripe.

Re: Degraded performance for multiple models

#130

Post by mysterydip »

bee_rider wrote: post #77
corvad wrote: post #46

Earlier quoted context omitted.

Wonder what Wednesday will be.

Power grid Thursday will be the rest of the infrastructure Then Friday we can turn off civilization for the weekend. Somebody remember to flip it back on Sunday night.

Reminds me of a company I worked at that paid for redundant power grids. One time the power went out and… nothing. The boss angrily calls up the power company and they tell him “Oh yeah, it’s a manual transfer switch. Bob is already on his way.” I think it took 15 minutes.
Post reply on HN