Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#1

Post by aarondong » Fri, Jul 24, 2026, 7:45 PM UTC

Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard
artificialanalysis.ai

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#2

Post by aarondong » Fri, Jul 24, 2026, 7:45 PM UTC

Before getting too excited, take a look at the intelligence vs cost matrix: https://artificialanalysis.ai/models?intelligence-index-toke...

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#3

Post by midnightbobarun » Fri, Jul 24, 2026, 7:55 PM UTC

aarondong wrote: post #2

Before getting too excited, take a look at the intelligence vs cost matrix: https://artificialanalysis.ai/models?intelligence-index-toke...

5.6 Sol (max) being cheaper than all of these is wild, considering how good the output is too

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#4

Post by claude-ai » Fri, Jul 24, 2026, 8:30 PM UTC

On my end, Opus 5 is Haiku level vs. Opus 4.8 (good) and Fable (superb).

Gets confused by permission prompts, cannot debug a failing test it caused (Opus 4.8 got it right after, without tens of rounds "thinking").

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#5

Post by Schiendelman » Fri, Jul 24, 2026, 9:25 PM UTC

midnightbobarun wrote: post #3

aarondong wrote: post #2

Before getting too excited, take a look at the intelligence vs cost matrix: https://artificialanalysis.ai/models?intelligence-index-toke...

5.6 Sol (max) being cheaper than all of these is wild, considering how good the output is too

This must be on API costs, not counting the $100/200 tiers, right?

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#6

Post by giancarlostoro » Fri, Jul 24, 2026, 10:20 PM UTC

midnightbobarun wrote: post #3

aarondong wrote: post #2

Before getting too excited, take a look at the intelligence vs cost matrix: https://artificialanalysis.ai/models?intelligence-index-toke...

5.6 Sol (max) being cheaper than all of these is wild, considering how good the output is too

Probably because they made ASICs to run inference for less.

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#7

Post by brookst » Fri, Jul 24, 2026, 10:21 PM UTC

giancarlostoro wrote: post #6

midnightbobarun wrote: post #3

Earlier quoted context omitted.

5.6 Sol (max) being cheaper than all of these is wild, considering how good the output is too

Probably because they made ASICs to run inference for less.

Are those actually deployed at scale yet?

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#8

Post by brcmthrowaway » Fri, Jul 24, 2026, 10:30 PM UTC

brookst wrote: post #7

giancarlostoro wrote: post #6

Earlier quoted context omitted.

Probably because they made ASICs to run inference for less.

Are those actually deployed at scale yet?

Yes.

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#9

Post by nijave » Fri, Jul 24, 2026, 10:33 PM UTC

midnightbobarun wrote: post #3

aarondong wrote: post #2

Before getting too excited, take a look at the intelligence vs cost matrix: https://artificialanalysis.ai/models?intelligence-index-toke...

5.6 Sol (max) being cheaper than all of these is wild, considering how good the output is too

I think on swebench verified luna was only like 3% points lower for 1/5 the cost

Like 96% vs 93% or something

Re: Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard

#10

Post by wmf » Fri, Jul 24, 2026, 10:44 PM UTC

brcmthrowaway wrote: post #8

brookst wrote: post #7

Earlier quoted context omitted.

Are those actually deployed at scale yet?

Yes.

I hate to disagree with Broadcom Throwaway himself but it's unlikely that the OpenAI Jalapeno ASIC has been deployed yet. It takes 6-12 months to test, develop software, ramp production, etc.