Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 5

anthropic.com

Post reply on HN
Print view View original

271–280 of 1001 posts

Re: Claude Opus 5

#271

Post by Dibes »

landrew_ wrote: post #224
Dibes wrote: post #104

I'm not sure what to make of this graph[0]. It shows medium as the most effective thinking mode by far for frontier code. It's the only case that I saw going through the system card where more reasoning effort meaningfully negatively impacted the resulting eval. I know sometimes max efforts show a small dip, but this is substantial. I wonder why in the world that is? [0] https://imgur.com/a/Nv8V7Ry

apparently it got docked points for editing files out of scope

Do you have a source for this? That would explain it, but could be a bit of a concern on the general focus the model at higher thinking exhibits.

Re: Claude Opus 5

#272

Post by LoganDark »

These cybersecurity safeguards are really annoying. There are ethical reasons to reverse-engineer and binary-patch software; for example Rewind got acquired by facebook and, as a gift to all their customers, implemented a killswitch in their software to ensure it will eventually stop functioning. I kept using a version without the killswitch, but the macOS 27 update killed it, and I needed binary patching to fix it. I should be allowed to repair software I purchased (I did purchase it like a month before they sold out), but unfortunately this overlaps significantly with cybersecurity.

Re: Claude Opus 5

#274

Post by orangecat »

bovermyer wrote: post #206

This stood out to me as a little concerning: > The model hallucinates factual claims slightly more than Opus 4.8, despite being more accurate overall.

That seems to be for the "AA-Omniscience" test where you get +1 for a correct answer, -1 for a wrong answer, and 0 for "I don't know". If a model is more than 50% confident in its answer, it should go ahead and submit it even though it will sometimes be wrong.

I'd be curious to see a version of the test where models are asked to give a probability that their answers are correct so we can see how calibrated they are.

Re: Claude Opus 5

#276

Post by [deleted] »

paxys wrote: post #41

Looking at all these releases it’s not a surprise that model routing is the fastest growing segment in AI right now. There are 10+ LLM companies, each with dozens of models of different modalities, each model with multiple size variants, then different “thinking” levels, then agentic modes, “pro” modes, a “fast” option, standard vs flex vs batch execution. And of course each end combination has a different input/outp…

[deleted]

Re: Claude Opus 5

#278

Post by jjcm »

acmnrs wrote: post #46

From the prompting guide https://platform.claude.com/docs/en/build-with-claude/prompt... >: > Claude Opus 5's default user-facing responses run longer than prior Opus models'. The benchmarks do show Opus 5 as slightly more expensive than 4.8, although the scores are much higher. This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their mod…

> especially with OpenAI making so much progress with the efficiency of their models

To be fair though, Sol tends to go off the rails sometimes. It's much less reliable than Fable in its outputs. It tends to be overzealous in its research/changes.

Re: Claude Opus 5

#279

Post by doctorpangloss »

postalcoder wrote: post #135

I think the most important thing here is not absolute performance. It's that organizations now have access to a Fable-ish model without Fable's 30-day data retention requirement[0]. > "Consistent with prior Opus models, Opus 5 does not have data retention requirements for general access."[1] On the Opus model release page, the reason why Fable doesn't have an ARC-AGI score is because of that retention policy[2]. 0: h…

do you mean, that organizations now have access to Fable-ish pelican drawing?
Post reply on HN