Live data from Hacker News

Claude Sonnet 5

anthropic.com

461–470 of 822 posts

Re: Claude Sonnet 5

#461

Earlier quoted context omitted.

"dangerous cyber skills, such as developing software exploits" is very plainly referring to the same thing you are, but is more precise industry terminology rather than the loaded slang "hack".

I was referring to "Lower ability to perform cybersecurity-related tasks," which is newspeak for hacking.

No, that is very intentionally referring to a broader set of things than "hacking".

Re: Claude Sonnet 5

#462
post #202

Important to note: "Sonnet 5 is an upgrade to Sonnet 4.6, but it uses an updated tokenizer that changes how the model processes text to improve performance (this is similar to the tokenizer change we introduced with Claude Opus 4.7). The tradeoff is that the same input can map to more tokens: roughly 1.0–1.35× depending on the content type. The introductory pricing is set so that the transition to Sonnet 5 is roughly…

[deleted]

Re: Claude Sonnet 5

#463

The cost per task chart is telling me that I should _never_ use Sonnet 5 above medium effort level - Opus always performs better for a given cost. So I guess the takeaway is that if Sonnet 5 medium isn't good enough for you, switch models, not effort levels.

I feel like the charts have been adjusted. I am quite sure, they looked different a couple hours ago...

They've absolutely both changed. The initial version I saw didn't include max effort data points on the first chart, and the plot itself was much less favorable to Sonnet at high/xhigh relative to Opus, but the new chart shows them as closer competitors. Weird.

Re: Claude Sonnet 5

#464

Earlier quoted context omitted.

> any American granny with access to Fable 5, Fable is effectively not available to the general public in the US either

True, but Trump & Co. did give them permission to let Americans continue using it, but Anthropic turned that down (for now at least...).

Because there is no practical way to comply with what they asked for. They'd have to start validating their users passports.

Re: Claude Sonnet 5

#465

Earlier quoted context omitted.

Your benchmark has Gemini 3.5 Flash as the best model, which doesn't compute for me

This guy had a terrible broken benchmark that gets hawked every release, and I wish HN would ban accounts that essentially exist to hawk a personally owned site, especially such a bad one.

If you were right, the karma system would largely take care of this. It really sounds like this is more of your personal view

Re: Claude Sonnet 5

#466
post #293
post #253

Earlier quoted context omitted.

Some napkin math -- total global labor compensation is about 50% of the GDP, which puts it in the USD 50 - 60 Trillion range: https://ourworldindata.org/grapher/labor-share-of-gdp This source claims that knowledge workers alone (probably because they are paid much more) account for 35 - 50 Trillion of that: https://github.com/danielmiessler/Substrate/blob/main/Data/K... If LLMs can boost their productivity even by an…

I want to propose alternative reality where 1.5-2.5T in value doesn't go to a handful of companies. Instead it turns out to be like restaurants where this gets distributed to lots and lots of small, local, mostly interchangeable teams. There will of course be some super star "chefs" leading the industry and setting trends and some "restaurant chain" like big businesses and supply chain for all of this.

FWIW I do think that availability of competitive open weight and other non-frontier models, along with improvements in harnesses that can get good results out of these models, will result in less concentration and a healthier marketplace.

However, these frontier labs are also making moves that could let them capture a disproportionate share of the upside. One possibility is a situation analogous to the smartphone manufacturing space, where there are dozens of players but just a handful (e.g. Apple, Samsung in smartphones) capture the lion's share of the revenue.

Re: Claude Sonnet 5

#467

Earlier quoted context omitted.

It's funny the exact same thing happened to Gemini 3.5 flash. Cheaper and more agentic model that ends up worse and more expensive than 3.5 pro low.

3.5 Pro not yet launched, you mean 3.1 pro?

Yes sorry for the typo

Re: Claude Sonnet 5

#468
post #422

Earlier quoted context omitted.

What’s everyone favorite GLM provider? z.ai doesnt always have the most reliable AI but I don’t mind the party seeing my trade secrets and thoughts compared to an American corporation + the party seeing my trade secrets and thoughts. So thats not a functional difference to me, and the Chinese one won’t reply to subpoenas so thats a value add tbh So I’ll consider all, fastest tokens/sec wins

Fireworks.ai is solid. And if you care more about speed than cost they have a "fast" variant that I think just throws more hardware at the model for about 2x the cost.

The privacy policy indicates that they track you and share your data to ad networks like Meta. Yikes.

Re: Claude Sonnet 5

#469

Earlier quoted context omitted.

This guy had a terrible broken benchmark that gets hawked every release, and I wish HN would ban accounts that essentially exist to hawk a personally owned site, especially such a bad one.

If you were right, the karma system would largely take care of this. It really sounds like this is more of your personal view

Karma systems are never perfect, and most people will not assume this is a pattern.

(ie. won't feel the need to downvote them just for having yet another crappy AI benchmark)

I only recognize it because I build a product that leaves me looking for information on every major release... and every major release a new crop of folks reply confused about the anomalies on top of anomalies that they're seeing, and they slowly learn this person is just way more unserious than the dogged distribution would imply.

Re: Claude Sonnet 5

#470

The cost per task chart is telling me that I should _never_ use Sonnet 5 above medium effort level - Opus always performs better for a given cost. So I guess the takeaway is that if Sonnet 5 medium isn't good enough for you, switch models, not effort levels.

While I appreciate, they publish this information, it's increasingly hard to keep track of it all. I've lost the mental model of how different models at different effort levels perform and what tasks they are good at. In practice, I tend to just use the default on Claude Code that works well enough. But I wonder to what degree other users really play around with these settings to optimize for their project.

Same advice as ever? We call it context engineering now, but prompt engineering still matters a lot. Most of the failures I run into are unspecified assumptions made by the model that derails the conversation, but usually updating the first prompt fixes it. Opus in my experience is a bit better about checking assumptions, while Sonnet will plow on ahead. An example is mentioning a file that doesn't exist: Sonnet will go ahead and try to grep your entire hard drive for it. Opus will say it's not local and request the path.

I trust neither for general knowledge and I still find Opus giving me answers that are completely BS. But the token spend for Q&A is nothing compared to coding, so I always use Opus + a lot of thinking. For coding, I find Opus to be better value/token but I haven't done any sort of rigorous test.

Post reply on HN