Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 5

anthropic.com

Post reply on HN
Print view View original

131–140 of 822 posts

Re: Claude Sonnet 5

#131

Post by ZeWaka »

doctoboggan wrote: post #29

The cost per task chart is telling me that I should _never_ use Sonnet 5 above medium effort level - Opus always performs better for a given cost. So I guess the takeaway is that if Sonnet 5 medium isn't good enough for you, switch models, not effort levels.

It's very interesting. Why even release a new product that underperforms at the same price level? Why not just lock it?

I guess it's probably a lot cheaper for them to run, and it cuts costs for them. Seems disingenuous, though.

Re: Claude Sonnet 5

#132

Post by sixtyj »

conradkay wrote: post #19

Wow, seems worse even on price/performance than GLM 5.2, which is only 744b parameters. From the system card: "On CyberGym vulnerability discovery, Claude Sonnet 5 is less capable than Sonnet 4.6, and far less capable than Opus 4.8 and Mythos 5 As with the other evaluations in this section, these results were achieved with all safeguards turned off. When run with our default mitigations, Sonnet 5 scored a 0 on CyberG…

I have tried to rewrite an article with GLM-5.2 and with Sonnet 4.6. Completely different results as LLM is non-deterministic. But GLM-5.2 made a lot of subtle mistakes that needed to be corrected by hand. On the opposite, Sonnet found and corrected all mistakes in the second round.

Similar situation was with planning and coding. GLM-5.2 seems to be good “on paper” but the real usage results was different.

And I am not an attorney for Claude or GLM-5.2… :)

But as I’ve been using LLM models daily since Nov 2022 I have realized that all common tests have to be confirmed in your project - there is no “one model rules them all” - you need to dig out a specific model from that LLM haystack with thousands of models.

Benchmarks help but they start to be similar to fuel consumption specs in car ads - real consumption is different for everybody :)

Re: Claude Sonnet 5

#134

Post by intellijdd »

doctoboggan wrote: post #29

The cost per task chart is telling me that I should _never_ use Sonnet 5 above medium effort level - Opus always performs better for a given cost. So I guess the takeaway is that if Sonnet 5 medium isn't good enough for you, switch models, not effort levels.

I noticed that as well but with the introductory pricing, I wonder how true that is.

It would be great to see these charts with the promotional pricing just because it’s here for about two whole months.

I guess I could get Sonnet 5 to do it.

Re: Claude Sonnet 5

#135

Post by popalchemist »

Brendinooo wrote: post #129
microtonal wrote: post #31

Claude Sonnet 5 is built to be the most agentic Sonnet model yet. It can make plans, use tools like browsers and terminals, and run autonomously at a level that, just a few months ago, required larger and more expensive models. I have been using Sonnet 4.6 more than Opus, because I'm mostly doing agent-assisted development and not fully agent-driven development. This announcement does not make me positive, I have fou…

Yeah, there's a real opportunity for one of these companies to invest time in a model that's tuned for, to use your term, agent-assisted developement. Trouble is, everyone inside their buildings seems to believe that no one will be working like that in a year or two.

Whether they believe it or not is immaterial. It is the end-goal they want to achieve, because then they own the means of production entirely.

Re: Claude Sonnet 5

#136

Post by epolanski »

thewebguyd wrote: post #126
epolanski wrote: post #120

Earlier quoted context omitted.

I've been saying for ages that since Opus 4.6 models are increasingly smarter but further unhelpful as assistants. Fable was amazing as a vibecoder but as an assistant it can't resist jumping into implementation and filling chats of pointless jargon. It's really grim if you're looking for assistance instead of an implementor. GPT 5.5 Pro and Fable are gorgeous bullshitters that pretend to be right (often convincingly…

By design, unfortunately. If they are just assistants, they can't sell the dream of "we're going to replace human labor completely" to the C-suite.

I think that they are simply evaluated on prompt to solution benchmarks.

Re: Claude Sonnet 5

#137

Post by baq »

thewebguyd wrote: post #126
epolanski wrote: post #120

Earlier quoted context omitted.

I've been saying for ages that since Opus 4.6 models are increasingly smarter but further unhelpful as assistants. Fable was amazing as a vibecoder but as an assistant it can't resist jumping into implementation and filling chats of pointless jargon. It's really grim if you're looking for assistance instead of an implementor. GPT 5.5 Pro and Fable are gorgeous bullshitters that pretend to be right (often convincingly…

By design, unfortunately. If they are just assistants, they can't sell the dream of "we're going to replace human labor completely" to the C-suite.

It isn’t a dream, it’s a reality for some of us here and it will be increasingly so for everyone else. Amazingly, USG intervening slowed the dynamic greatly (fortunately?)

The problem is obviously who will be left. There’s a lot of scifi to catch up on.

Re: Claude Sonnet 5

#138

Post by AquinasCoder »

doctoboggan wrote: post #29

The cost per task chart is telling me that I should _never_ use Sonnet 5 above medium effort level - Opus always performs better for a given cost. So I guess the takeaway is that if Sonnet 5 medium isn't good enough for you, switch models, not effort levels.

While I appreciate, they publish this information, it's increasingly hard to keep track of it all. I've lost the mental model of how different models at different effort levels perform and what tasks they are good at.

In practice, I tend to just use the default on Claude Code that works well enough. But I wonder to what degree other users really play around with these settings to optimize for their project.

Re: Claude Sonnet 5

#140

Post by thewebguyd »

phillipcarter wrote: post #12

Seems to be another great incremental update to the workhorse, nice! I've been using Sonnet instead of Opus for almost all coding tasks for a while now. A little elbow grease to break down tasks and you can spend a lot less money for just about the same output quality.

Yeah I think people are sleeping on the smaller/faster models like Sonnet. As long as you have a detailed plan or small, well scoped individual tasks Sonnet can implement just fine. Opus will still do better at more open ended tasks or completely "vibe coding." Or spec/plan with Opus, and have Sonnet implement.
Post reply on HN