Qwen3.8 Max now ranked as the best overall model by agentic index
artificialanalysis.ai
Qwen3.8 Max now ranked as the best overall model by agentic index
1–10 of 364 posts
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#2Strange that the page https://artificialanalysis.ai/agents/coding-agents doesn't even mention "Qwen" once if it's now the "best" according to one of their one index?
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#3I am so excited for Qwen 3.8 27B. It’s a shame how slow prefill (~3-400) is on a strix halo but it’s such a good model for agentic tasks.
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#4Why does an open weights model cost nearly the same as GPT5.6? $1.14 vs $1.23 on the cost index. Since you can't presumably run this on your own hardware given the model size and hence gain other things like privacy, I don't see any reason to move away from GPT at this rate.
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#5I believe it. It's extremely good at troubleshooting. I gave Qwen and Kimi K3 the same annoying, complicated, intermittent bug to track down. Kimi did a bit better in understanding the existing code, but Qwen built some diagnostic tools and did an excellent statistical analysis on the log data. Qwen got way closer to the truth.
I'm very much looking forward to their forthcoming smaller model Qwen 3.8 releases. A version that can easily run locally would be great.
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#6Why does an open weights model cost nearly the same as GPT5.6? $1.14 vs $1.23 on the cost index. Since you can't presumably run this on your own hardware given the model size and hence gain other things like privacy, I don't see any reason to move away from GPT at this rate.
It's not enough that it's better?
Many providers will host it and will compete on price. It also can't easily be taken away because one company (or one government) decides they don't want it around any more. People can fine-tune it for particular workloads.
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#7Go China!
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#8I find that surprising.
I've been trying it on several projects and have found it's pretty sloppy. It leaves stuff broken, doesn't reliably write tests to check its own work unless explicitly prompted, misunderstands the assignment, etc.
It is smart and reasonably quick but not reliable.
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#9Is there a path to distill this model to do very specific things? Like a RAG strategy for a small (or even large) corpus?
Re: Qwen3.8 Max now ranked as the best overall model by agentic index
#10Strange that the page https://artificialanalysis.ai/agents/coding-agents doesn't even mention "Qwen" once if it's now the "best" according to one of their one index?
Different benchmarks:
> Artificial Analysis Agentic Index: Represents the weighted average of agentic capabilities benchmarks in the Artificial Analysis Intelligence Index (GDPval-AA v2, Tau³-Banking)
> Artificial Analysis Coding Agent Index v1.3 incorporates 3 benchmarks: DeepSWE, Terminal-Bench v2, and SWE-Atlas-QnA
Qwen3.8 Max is 55.4 on the Agentic Index but hasn't been tested for the Coding Agent Index.