Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.8 Max now ranked as the best overall model by agentic index

artificialanalysis.ai

Post reply on HN
Print view View original

151–160 of 364 posts

Re: Qwen3.8 Max now ranked as the best overall model by agentic index

#151

Post by SwellJoe »

ux266478 wrote: post #80
drnick1 wrote: post #42

Earlier quoted context omitted.

> It's not enough that it's better? It's barely better, and barely cheaper, not really enough to challenge the status quo IMO. Half the price for basically the same performance would be a much stronger value proposition.

What status quo? Just look at Openrouter's rankings: https://openrouter.ai/rankings Things change radically month to month. Nobody is remotely close to capturing the market or having any kind of stability over time. People move around quite a lot, often to sidegrade within a generation. Just playing fly on the wall with discourse would be enough to tell you all of this, even without the data to back it up.

If anybody has, it's DeepSeek. But, with the promised price hikes, I'm sure that'll change. I'm guessing they're raising prices not because they're not making a profit at those prices, but because they're running into capacity problems and need to slow down until they've got more or risk providing poor service. For now DeepSeek Flash is the best deal going for API usage and its popularity makes sense.

Also, OpenRouter misses most of the usage of the US models, as most people are getting those from the vendor directly via subscriptions.

Re: Qwen3.8 Max now ranked as the best overall model by agentic index

#152

Post by kyxsc »

brcmthrowaway wrote: post #18

Could someone like Apple be playing the long game - Good Enough(tm) intelligence will eventually fit in our pocket and homes?

Apple is already doing this... they worked with Gemini to distill the model into a smaller one that fits on your phone. If you have iOS 27 Beta, you're already using this

Re: Qwen3.8 Max now ranked as the best overall model by agentic index

#153

Post by [deleted] »

ericd wrote: post #102
AlexandrB wrote: post #99

Earlier quoted context omitted.

VCs are footing the bill for that $200 subscription.

They have something like 80% gross margins, are at a $100B/yr ARR, and are growing at 10x per year... If that keeps up, they're going to be doing more revenue than Google in a year ($400B ARR, 20% per year growth)

[deleted]

Re: Qwen3.8 Max now ranked as the best overall model by agentic index

#154

Post by aenis »

tempest_ wrote: post #95
aenis wrote: post #84

Earlier quoted context omitted.

I managed to lose around $300 in credits I had saved for some emergency /fast sessions the following way: switch to Fable. Work on the design. Downgrade to Opus for the build. If any of other parallel Opus session has /fast enabled it seems to enable it for the newly spawned session by default. Before I knew it, the $300 was gone. I think the bug is now solved, but it was rather unpleasant. I dont ever remember bugs…

I dont love it. Opus 5 is just a token burner. I use fable plan and spawn opus 4.8 workflows which seems to work alright.

I suspect it must depend on how one manages their codebase - wrt to docs, ADRs, and general guardrails.

For me it is not great for design work - Fable is way better, and 4.8 was conservative and thus better (Opus 5 seems to jump to conclusions far more eagerly). But for overnight builds, where I give it 8hrs worth of work on LLDs created by Fable - its great. Where Opus 4.8 would often lose the plot and stop for questions clearly answered in the LLD - Opus 5 does manage to complete. Since it launched, I don't remember it ever disappointing me with builds. But designs? Boy, is this thing explosively stupid sometimes.

Re: Qwen3.8 Max now ranked as the best overall model by agentic index

#155

Post by swalsh »

hahahaa wrote: post #146
swalsh wrote: post #141

Earlier quoted context omitted.

Its tough to go from max account at home and pay per usage enterprise account at work with heavy usage limits... but the limits are there because pricing is insane. Feel like I'm in the $5 Uber rides phase at home.

The Chinese models are the public transport in the uber analogy. Once the price the goes up catch the bus!

Lol perfect analogy. I'm still paying for claude because the quality is unmatched.

Re: Qwen3.8 Max now ranked as the best overall model by agentic index

#156

Post by tarnith »

theropost wrote: post #60

Anthropic is a bit nuts, I had $260 of credits on my max account for the extra usage the other night. It was expiring, so I figured I'll fire up an agentic swarm to deep dive and make some deep changes to some old cold bases.. literally 25 minutes or less, $260 burnt, it didn't get get into the implementation, just wrote a ton of useless plans for the most part. It really opened my eyes to what they expect to charge…

Hint: The new models are really good at burning tokens.

I've had to use it a bit for work, and it's been remarkable watching the degradation in performance with the default suggested current models (Opus 5 as a prime example) vs the models that got them huge attention a year ago (Opus 4.6)

If you give 4.6 a spec, or existing code to implement a feature in, it will ask some pointed questions if there's something unclear in the spec, and then produce a plan and move to implement it.

5 will freak out at even a basic task, ask itself if it's own assumptions or your instructions are correct, proceed to re-assess it's own plan, and it's instructions 3-4 times, and then maybe produce code after burning several hundred thousand tokens (and quite a bit of time) analyzing existing code and thoroughly sweeping it for irrelevant problems both to the task it was given and the spec it came up with.

It's quite bizarre to me how well advertised the benchmarks and anecdotes from people one shotting MVP browser games are, compared to the experience of everyone I know that's had to actually use it to accomplish even a relatively basic task.

Re: Qwen3.8 Max now ranked as the best overall model by agentic index

#157

Post by jiggawatts »

d4rkp4ttern wrote: post #116
seizethecheese wrote: post #97

Opus is still first in Intelligence Index followed by Fable, GPT 5.6, Kimi K3 then Qwen 3.8 max. https://artificialanalysis.ai/#intelligence Our leaderboard combines Arena ELO, AA Intelligence index, latency and speed and goes: #1 Opus 5 #2 Kimi K3 #3 Qwen3.8 Max #4 GPT 5.6 Sol Source: http://pellmell.ai/leaderboard . This jumps around a lot based on the top throughput and latency of whatever provider happens to be b…

All these "intelligence" benchmarks miss something extremely important when using an LLM in a code-agent harness: How it communicates with you about what it did. Opus-5 is practically unusable (for complex tasks) in this sense - its updates are voluminous, and dense with cryptic language (there are numerous reddit threads complaining about this, so it's not just me). I often have to ask it to re-state concisely in pl…

GPT 5.6 has similar language quirks that makes its comments nearly unusable.

I wonder if this is a side effect of MoE models — they can write excellent prose, but not simultaneously with writing code.

Re: Qwen3.8 Max now ranked as the best overall model by agentic index

#158

Post by cesarvarela »

onomojo wrote: post #34

Any benchmark showing Opus 5 as the best just loses credibility for me. Anyone who's actually used Opus 5 daily knows what I'm talking about.

It is infuriating to interact with, but it is also first in many blind test leaderboards on LLMArena

Re: Qwen3.8 Max now ranked as the best overall model by agentic index

#159

Post by CuriouslyC »

onomojo wrote: post #34

Any benchmark showing Opus 5 as the best just loses credibility for me. Anyone who's actually used Opus 5 daily knows what I'm talking about.

Ironically, Opus 5 is the most benchmaxxed model I've seen from Anthropic. It is legitimately smart in a lot of ways but it has communication issues, both in terms of how it communicates (all the autism of GPT class models, without the brevity) and how well it catches all the nuance of what you tell it.

Re: Qwen3.8 Max now ranked as the best overall model by agentic index

#160

Post by arjie »

brettgo1 wrote: post #122

Out of curiosity, what's currently the best model I can use locally?

$500k - Kimi K3 (maybe $250k? Haven’t done this one)

$25k - DSv4 Flash

$4k - Qwen 3.6 35A3B Q5

$1k - Qwen 3.6 27B Q4

Some people prefer the sense over the MoE YMMV.

Post reply on HN