Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 5

anthropic.com

Post reply on HN
Print view View original

101–110 of 1001 posts

Re: Claude Opus 5

#101

Post by edumucelli »

alansaber wrote: post #56
acmnrs wrote: post #46

From the prompting guide https://platform.claude.com/docs/en/build-with-claude/prompt... >: > Claude Opus 5's default user-facing responses run longer than prior Opus models'. The benchmarks do show Opus 5 as slightly more expensive than 4.8, although the scores are much higher. This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their mod…

Is that true? Sol responses are also longer than prior models.

Anecdata: my workflow has been working on the same personal projects for months now with Codex. I cannot anymore finish my daily/weekly code with 4.8 anymore.

I was dividing my work between Codex and DeepSeek. Now I barely use DeepSeek, or never because Codex quota is enough after Sol

Re: Claude Opus 5

#102

Post by sambaumann »

varispeed wrote: post #34
vidarh wrote: post #18

Earlier quoted context omitted.

Fable is twice the price.

If Fable gets correct answer quicker, then you might pay less than doing back and forth with Opus, plus you lose more of your own time. I see no reason for using less able models in my workflows. There is this saying, penny wise and pound foolish

The CursorBench plot, for example, shows that fable does have slightly better performance, but Opus is pretty close, and is less expensive per task

Re: Claude Opus 5

#103

Post by albert_e »

Judging by the pace at which new models are released these days -- it feels like a Windows KB or VS Code patch release now.

Older models must be getting deprecated at the same (or faster) pace. So anything you built 3 months ago is probably going to break soon.

AI solutions need better insurance around model deprecation. Commercial API-only models that complete the full cycle from SOTA / gated-preview to unsupported and deprectated in a matter of months -- is no way to build serious software!

Re: Claude Opus 5

#104

Post by Dibes »

I'm not sure what to make of this graph[0]. It shows medium as the most effective thinking mode by far for frontier code.

It's the only case that I saw going through the system card where more reasoning effort meaningfully negatively impacted the resulting eval. I know sometimes max efforts show a small dip, but this is substantial. I wonder why in the world that is?

[0] https://imgur.com/a/Nv8V7Ry

Re: Claude Opus 5

#106

Post by Ancalagon »

nerdsniper wrote: post #65

Edit: It was pointed out to me that Opus 4.8 got "21%" for successfully fully completing ~1-in-5 tasks, but also got "55.7%" for obtaining significant partial credit on some of the ~4-in-5 tasks it could not fully complete. --------------- Why does Anthropic say here that Opus 4.8 scored 55.7% on OSWorld 2.0 benchmark, but the paper published by the authors of OSWorld 2.0 say they achieved a benchmark of ~21% with Op…

Its slop all the way down.

Re: Claude Opus 5

#107

Post by frenchie4111 »

acmnrs wrote: post #46

From the prompting guide https://platform.claude.com/docs/en/build-with-claude/prompt... >: > Claude Opus 5's default user-facing responses run longer than prior Opus models'. The benchmarks do show Opus 5 as slightly more expensive than 4.8, although the scores are much higher. This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their mod…

It's a step in the wrong direction but also token efficiency has become a focus relatively recently (just the past few weeks it seems like the zeitgeist has turned it's attention to efficiency) while work on this model probably started many many months ago. I would expect to see models released that focus on token efficiency in 6-12 months

Re: Claude Opus 5

#108

Post by modeless »

Wow, 30% on ARC-AGI-3 for $20k total. Huge jump from GPT-5.6's 7.8% at $20k per task. I continue to believe ARC-AGI measures something different and important compared to other benchmarks.

Re: Claude Opus 5

#109

Post by lukeschlather »

acmnrs wrote: post #46

From the prompting guide https://platform.claude.com/docs/en/build-with-claude/prompt... >: > Claude Opus 5's default user-facing responses run longer than prior Opus models'. The benchmarks do show Opus 5 as slightly more expensive than 4.8, although the scores are much higher. This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their mod…

On a small, easily digestible task, I compared Fable to Opus and the cost of Fable was easily 2x despite being fewer tokens, and the output was not really better. Obviously, there are tasks where using Fable matters but honestly they're rather unusual. And for a lot of tasks I've found downgrading to Sonnet can be valuable because Fable and Opus are a lot more secretive about what they're doing, and it's impossible to "listen to them think" and stop them when they start making off-the-wall inferences/assumptions and going down bad paths.

I think in the long run tokens are probably the wrong thing; it's compute and cache memory that you need to be measuring, and when you look at it that way I suspect in most cases the models have pretty similar performance.

Re: Claude Opus 5

#110

Post by ceejayoz »

shwaj wrote: post #97
rb2e wrote: post #17

https://www.anthropic.com/news/claude-opus-5 - A blog post for those not wanting to go through a 190ish page pdf

I like how they highlighted Opus 5 as the best for “Agentic Coding” even though the number is slightly lower than Fable. Close enough for marketing, I guess!

Best can describe multiple things.

Almost as good for half the cost is something I'm very comfortable describing that way.

Post reply on HN