It doesn’t even show up in the raw intelligence index, so how could it possibly be the best?
Qwen3.8 Max now ranked as the best overall model by agentic index
21–30 of 364 posts
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#22Earlier quoted context omitted.
For one thing, providers of open models can't arbitrarily increase their prices without facing competition.
But given the extremely low cost of switching, why wouldn't you use the cheaper one if they're comparable?
As low as it is, switching between providers on OpenRouter is still lower.
That said, it's a fair point. For me, it boils down to things covered here: https://earendil.com/posts/session-portability/
Things like obscured reasoning traces.
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#23Strange that the page https://artificialanalysis.ai/agents/coding-agents doesn't even mention "Qwen" once if it's now the "best" according to one of their one index?
Indeed, and this Qwen 3.8 max specific page: https://artificialanalysis.ai/models/qwen3-8-max Doesn't have the claim either. Clickbait?
This page has it, scroll to "Intelligence" header (not the highlights one, but second on the page / with black square) and click "Agentic Index"
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#24I find that surprising. I've been trying it on several projects and have found it's pretty sloppy. It leaves stuff broken, doesn't reliably write tests to check its own work unless explicitly prompted, misunderstands the assignment, etc. It is smart and reasonably quick but not reliable.
It’s because they’re doing some sort of combined score of intelligence, speed and cost. On pure intelligence it doesn’t even show up in the top 10.
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#25Why does an open weights model cost nearly the same as GPT5.6? $1.14 vs $1.23 on the cost index. Since you can't presumably run this on your own hardware given the model size and hence gain other things like privacy, I don't see any reason to move away from GPT at this rate.
it's a big honking trillion some parameters model. it's not cheap to run
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#26A couple days ago they had published an overall score of 53 for this model, but that was removed and today it returned with a score of 56.
I wasn't able to find an explanation from them. Anyone knows what happened?
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#27Is there a path to distill this model to do very specific things? Like a RAG strategy for a small (or even large) corpus?
Depends on what you want to do. Some task specific models can be trained with a few ten or hundred thousand training examples so you can use a bigger model to produce synthetic training examples and then fine tune a smaller student model. I think that's the usual process. Whether you'd get acceptable performance this way depends, as mentioned, on what you're trying to do and what you'd consider acceptable.
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#28Strange that the page https://artificialanalysis.ai/agents/coding-agents doesn't even mention "Qwen" once if it's now the "best" according to one of their one index?
They didn't run all benchmarks. It's the best in AA agentic index (GDPval-AA v2, ³-Banking) but not coding index (DeepSWE which is missing, Terminal-Bench v2.1 they have 81% vs 90% for Sol, SWE-Atlas-QnA missing).
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#29I am so excited for Qwen 3.8 27B. It’s a shame how slow prefill (~3-400) is on a strix halo but it’s such a good model for agentic tasks.
How are you running it on a Strix Halo? The weights aren't out yet, are they?
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#30I believe it. It's extremely good at troubleshooting. I gave Qwen and Kimi K3 the same annoying, complicated, intermittent bug to track down. Kimi did a bit better in understanding the existing code, but Qwen built some diagnostic tools and did an excellent statistical analysis on the log data. Qwen got way closer to the truth. I'm very much looking forward to their forthcoming smaller model Qwen 3.8 releases. A vers…
How CLI are you guys using for qwen and kimi?