Live data from Hacker News

Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

artificialanalysis.ai

81–90 of 251 posts

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#81

The more interesting finding is that it's still the second most expensive model (after Fable 5) by a long shot. At least two models (GPT-5.6, Kimi K3) match its score (~1-2% diff) for half the cost.

The chart shows max effort, used mostly by price-insensitive enterprise users. At medium effort it drops to almost half K3’s cost, and is probably sufficient for 95% of coding tasks.

Opus medium = Sol high = 56, but still 25% more expensive

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#82
post #22

Earlier quoted context omitted.

What are you asking that you’re so regularly running into censorship?

The other day, I told claude that my physical wifi door unlock push buttons is a security risk because someone could run away with it and then unlock the door from outside whenever he wants. Then I told it that I want to introduce a concept of public/private key to uniquely identify my push buttons so that I can disable them individually using some crypto like ed25519... Fable understood it as something along the lin…

> Fable understood it as

The dumbfuck bouncer Anthropic put in front of Fable decided this.

Fable is a PR model. It’s great. But if it were an employee, it would be the brilliant one who regularly shows up to work high. Not useless. But not reliable.

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#83
post #22

Earlier quoted context omitted.

What are you asking that you’re so regularly running into censorship?

Things Fable's classifier has flagged, a non-exhaustive list, – "Does collagen supplementation empirically work?" - "Can you help me figure out how to calculate and generate Kaplan-Meier curve?" – "Why do rabbits reproduce so frequently?" — "Can you tell me how collagen peptides are absorbed by my digestive tract and the role they play? Can you teach me [edit: how] this works at the biomolecular level?"

Oh biology! It's dangerous!

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#84
post #23
post #17

It's new, normal.

Wait until DeepSeek v5 Pro is released in less than a month and costs 1/100 to perform the same tasks. "Not fair! They distilled Opus 5!"

Why doesn't Anthropic distil themselves then if that's what makes it cheaper. IMO the cost savings can't all be down to distillation.

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#85

The more interesting finding is that it's still the second most expensive model (after Fable 5) by a long shot. At least two models (GPT-5.6, Kimi K3) match its score (~1-2% diff) for half the cost.

The chart shows max effort, used mostly by price-insensitive enterprise users. At medium effort it drops to almost half K3’s cost, and is probably sufficient for 95% of coding tasks.

For a fair comparison, you should compare to K3 (which AA has not tested yet unfortunately) and GPT 5.6 Sol also on medium or the closest equivalent

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#87
post #42
post #14

Very interesting that one of the components is "AA-Omniscience Index" AA-Omniscience Index (higher is better) measures knowledge reliability and hallucination. It rewards correct answers, penalizes hallucinations, and has no penalty for refusing to answer. This seems to be a good proxy for param size/density and the ranking breaks down as such: Claude Fable 5 (with fallback), Gemini 3.1 Pro Preview, Claude Opus 5 (Ma…

Gemini 3.1 pro is really good for knowledge tasks. Google has done well there. And image analysis with Gemini flash 3.6 is solid. It’s just anything coding or agentic they fall short.

they likely tune their models for areas where they have their money: search, ads, youtube, etc.

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#88

Earlier quoted context omitted.

The other day, I told claude that my physical wifi door unlock push buttons is a security risk because someone could run away with it and then unlock the door from outside whenever he wants. Then I told it that I want to introduce a concept of public/private key to uniquely identify my push buttons so that I can disable them individually using some crypto like ed25519... Fable understood it as something along the lin…

> Fable understood it as The dumbfuck bouncer Anthropic put in front of Fable decided this. Fable is a PR model. It’s great. But if it were an employee, it would be the brilliant one who regularly shows up to work high. Not useless. But not reliable.

[dead]

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#90

Earlier quoted context omitted.

Are you using Claude Code/CoWork or an API client? I’m curious if it has different training that makes it more effective with specific instructions/ tool calling methods that are only implemented in official harnesses.

I'm curious about this too, and it's difficult to get any information about this given everyone has different setups, workflows and use-cases. I bizarrely had Opus 4.8 this week (in pi.dev within a podman container, using openrouter) start installing various python packages (and uv!) within the environment (not as root) when I asked it to code review some fairly basic Rust .rs files that were generally stand-alone (i…

There's probably some system prompt crap in claude code that tamps down that behavior? I wonder what it was even trying to do.
Post reply on HN