Is Qwen next architecture ironed out in llama cpp?
Qwen3-Coder-Next
31–40 of 443 posts
Re: Qwen3-Coder-Next
#32[dead]
Re: Qwen3-Coder-Next
#33It’s hard to elaborate just how wild this model might be if it performs as claimed. The claims are this can perform close to Sonnet 4.5 for assisted coding (SWE bench) while using only 3B active parameters. This is obscenely small for the claimed performance.
If it sounds too good to be true…
Should be possible with optimised models, just drop all "generic" stuff and focus on coding performance.
There's no reason for a coding model to contain all of ao3 and wikipedia =)
Re: Qwen3-Coder-Next
#34how can anyone keep up with all these releases... what's next? Sonnet 5?
Re: Qwen3-Coder-Next
#35The agent orchestration point from vessenes is interesting - using faster, smaller models for routine tasks while reserving frontier models for complex reasoning. In practice, I've found the economics work like this: 1. Code generation (boilerplate, tests, migrations) - smaller models are fine, and latency matters more than peak capability 2. Architecture decisions, debugging subtle issues - worth the cost of frontie…
I find it really surprising that you’re fine with low end models for coding - I went through a lot of open-weights models, local and "local", and I consistently found the results underwhelming. The glm-4.7 was the smallest model I found to be somewhat reliable, but that’s a sizable 350b and stretches the definition of local-as-in-at-home.
You're replying to a bot, fyi :)
Re: Qwen3-Coder-Next
#36Earlier quoted context omitted.
By the time that happens, Opus 5 and GPT-5.5 will be out. At that point will a GPT-5.2 tier open-weights model feel "good enough"? Based on my experience with frontier models, once you get a taste of the latest and greatest it's very hard to go back to a less capable model, even if that less capable model would have been SOTA 9 months ago.
It feels like the gap between open weight and closed weight models is closing though.
Mode like open local models are becoming "good enough".
I got stuff done with Sonnet 3.7 just fine, it did need a bunch of babysitting, but still it was a net positive to productivity. Now local models are at that level, closing up on the current SOTA.
When "anyone" can run an Opus 4.5 level model at home, we're going to be getting diminishing returns from closed online-only models.
Re: Qwen3-Coder-Next
#37Earlier quoted context omitted.
Essentially the more turns you have the more the agent is likely to fail since the error compounds per turn. Agentic model are tuned for “long horizon tasks” ie being able to go many many turns on the same problem without failing.
Much appreciated, but I mean more around "what do the error bars in the figure represent" than what the turn scaling itself is.
For the tasks in SWE-Bench Pro they obtained a distribution of agent turns, summarized as the box plot. The box likely describes the inter-quartile range while the whiskers describe the some other range. You'd have to read their report to be sure. https://en.wikipedia.org/wiki/Box_plot
Re: Qwen3-Coder-Next
#38Earlier quoted context omitted.
Essentially the more turns you have the more the agent is likely to fail since the error compounds per turn. Agentic model are tuned for “long horizon tasks” ie being able to go many many turns on the same problem without failing.
Much appreciated, but I mean more around "what do the error bars in the figure represent" than what the turn scaling itself is.
That's a box plot, so those are not error bars but a visualization of the distribution of a metric (min, max, median, 25th percentile, 75th percentile).
The benchmark consists of a bunch of tasks. The chart shows the distribution of the number of turns taken over all those tasks.
Re: Qwen3-Coder-Next
#39Earlier quoted context omitted.
By the time that happens, Opus 5 and GPT-5.5 will be out. At that point will a GPT-5.2 tier open-weights model feel "good enough"? Based on my experience with frontier models, once you get a taste of the latest and greatest it's very hard to go back to a less capable model, even if that less capable model would have been SOTA 9 months ago.
I think it depends on what you use it for. Coding, where time is money? You probably want the Good Shit, but also want decent open weights models to keep prices sane rather than sama’s 20k/month nonsense. Something like a basic sentiment analysis? You can get good results out of a 30b MoE that runs at good pace on a midrange laptop. Researching things online with many sources and decent results I’d expect to be doabl…
What does it mean for U.S. AI firms if the new equilibrium is devs running open models on local hardware?
Re: Qwen3-Coder-Next
#40Can anyone help me understand the "Number of Agent Turns" vs "SWE-Bench Pro (%)" figure? I.e. what does the spread of Qwen3-Coder-Next from ~50 to ~280 agent turns represent for a fixed score of 44.3%: that sometimes it takes that spread of agent turns to achieve said fixed score for the given model?
SWE-Bench Pro consists of 1865 tasks. https://arxiv.org/abs/2509.16941 Qwen3-Coder-Next solved 44.3% (826 or 827) of these tasks. To solve a single task, it took between ≈50 and ≈280 agent turns, ≈150 on average. In other words, a single pass through the dataset took ≈280000 agent turns. Kimi-K2.5 solved ≈84 fewer tasks, but also only took about a third as many agent turns.