Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-Coder-Next

qwen.ai

Post reply on HN
Print view View original

261–270 of 443 posts

Re: Qwen3-Coder-Next

#261

Post by kybernetikos »

johnsmith1840 wrote: post #203
dehrmann wrote: post #47

Earlier quoted context omitted.

I wonder if the future in ~5 years is almost all local models? High-end computers and GPUs can already do it for decent models, but not sota models. 5 years is enough time to ramp up memory production, consumers to level-up their hardware, and models to optimize down to lower-end hardware while still being really good.

Opensource or local models will always heavily lag frontier. Who pays for a free model? GPU training isn't free! I remember early on people saying 100B+ models will run on your phone like nowish. They were completely wrong and I don't think it's going to ever really change. People always will want the fastest, best, easiest setup method. "Good enough" massively changes when your marketing team is managing k8s cluster…

Gpt3.5 as used in the first commercially available chat gpt is believed to be hundreds of billions of parameters. There are now models I can run on my phone that feel like they have similar levels of capability.

Phones are never going to run the largest models locally because they just don't have the size, but we're seeing improvements in capability at small sizes over time that mean that you can run a model on your phone now that would have required hundreds of billions of parameters less than 6 years ago.

Re: Qwen3-Coder-Next

#262

Post by margalabargala »

Aurornis wrote: post #248
skhameneh wrote: post #23

It’s hard to elaborate just how wild this model might be if it performs as claimed. The claims are this can perform close to Sonnet 4.5 for assisted coding (SWE bench) while using only 3B active parameters. This is obscenely small for the claimed performance.

I experimented with the Q2 and Q4 quants. First impression is that it's amazing we can run this locally, but it's definitely not at Sonnet 4.5 level at all. Even for my usual toy coding problems it would get simple things wrong and require some poking to get to it. A few times it got stuck in thinking loops and I had to cancel prompts. This was using the recommended settings from the unsloth repository. It's always p…

Wonder where it falls on the Sonnet 3.7/4.0/4.5 continuum.

3.7 was not all that great. 4 was decent for specific things, especially self contained stuff like tests, but couldn't do a good job with more complex work. 4.5 is now excellent at many things.

If it's around the perf of 3.7, that's interesting but not amazing. If it's around 4, that's useful.

Re: Qwen3-Coder-Next

#263

Post by technotony »

simonw wrote: post #173

I got this running locally using llama.cpp from Homebrew and the Unsloth quantized model like this: brew upgrade llama.cpp # or brew install if you don't have it yet Then: llama-cli \ -hf unsloth/Qwen3-Coder-Next-GGUF:UD-Q4_K_XL \ --fit on \ --seed 3407 \ --temp 1.0 \ --top-p 0.95 \ --min-p 0.01 \ --top-k 40 \ --jinja That opened a CLI interface. For a web UI on port 8080 along with an OpenAI chat completions compati…

what are your impressions?

Re: Qwen3-Coder-Next

#264

Post by cmrdporcupine »

littlestymaar wrote: post #241
Robdel12 wrote: post #102

I really really want local or self hosted models to work. But my experience is they’re not really even close to the closed paid models. Does anyone any experience with these and is this release actually workable in practice?

> But my experience is they’re not really even close to the closed paid models. They are usually as good as the flagship model for 12-18 months ago. Which may sound like a massive difference, because somehow it is, but it's also fairly reasonable, you don't need to live to the bleeding edge.

And it's worth pointing out that Claude Code now dispatches "subagents" from Opus->Sonnet and Opus->Haiku ... all the time, depending on the problem.

Running this thing locally on my Spark with 4-bit quant I'm getting 30-35 tokens/sec in opencode but it doesn't feel any "stupider" than Haiku, that's for sure. Haiku can be dumb as a post. This thing is smarter than that.

It feels somewhere around Sonnet 4 level, and I am finding it genuinely useful at 4-bit even. Though I have paid subscriptions elsewhere, so I doubt I'll actually use it much.

I could see configuration OpenCode somehow to use paid Kimi 2.5 or Gemini for the planning/analysis & compaction, and this for the task execution. It seems entirely competent.

Re: Qwen3-Coder-Next

#265

Post by Keats »

mirekrusin wrote: post #251
Keats wrote: post #204

Earlier quoted context omitted.

Is there some indication on how the different bit quantization affect performance? IE I have a 5090 + 96GB so I want to get the best possible model but I don't care about getting 2% better perf if I only get 5 tok/s.

It takes download time + 1 minute to test speed yourself, you can try different quants, it's hard to write down a table because it depends on your system ie. ram clock etc. if you go out of gpu. I guess it would make sense to have something like max context size/quants that fit fully on common configs with gpus, dual gpus, unified ram on mac etc.

Testing speed is easy yes, I'm mostly wondering about the quality difference between Q6 vs Q8_K_XL for example.

Re: Qwen3-Coder-Next

#266

Post by cmrdporcupine »

orliesaurus wrote: post #34

how can anyone keep up with all these releases... what's next? Sonnet 5?

This is going to be a crazy month because the Chinese labs are all trying to get their releases out prior to their holidays (Lunar New Year / Spring Festival).

So we've seen a series of big ones already -- GLM 4.7 Flash, Kimi 2.5, StepFun 3.5, and now this. Still to come is likely a new DeepSeek model, which could be exciting.

And then I expect the Big3, OpenAI/Google/Anthropic will try to clog the airspace at the same time, to get in front of the potential competition.

Re: Qwen3-Coder-Next

#267

Post by CamperBob2 »

danielhanchen wrote: post #2

For those interested, made some Dynamic Unsloth GGUFs for local deployment at https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF and made a guide on using Claude Code / Codex locally: https://unsloth.ai/docs/models/qwen3-coder-next

Good results with your Q8_0 version on 96GB RTX 6000 Blackwell. It one-shotted the Flappy Bird game and also wrote a good Wordle clone in four shots, all at over 60 tps. Thanks!

Is your Q8_0 file the same as the one hosted directly on the Qwen GGUF page?

Re: Qwen3-Coder-Next

#268

Post by kylehotchkiss »

Is there any online resource tracking local model capability on say... a $2000 64gb memory Mac Mini? I'm getting increasingly excited about the local model space because it offers us a future where we can benefit from LLMs without having to listen to tech CEOs saber rattle about removing America of its jobs so they can get the next fundraising round sorted

Re: Qwen3-Coder-Next

#269

Post by nubg »

simonw wrote: post #173

I got this running locally using llama.cpp from Homebrew and the Unsloth quantized model like this: brew upgrade llama.cpp # or brew install if you don't have it yet Then: llama-cli \ -hf unsloth/Qwen3-Coder-Next-GGUF:UD-Q4_K_XL \ --fit on \ --seed 3407 \ --temp 1.0 \ --top-p 0.95 \ --min-p 0.01 \ --top-k 40 \ --jinja That opened a CLI interface. For a web UI on port 8080 along with an OpenAI chat completions compati…

what's the token per seconds speed?

Re: Qwen3-Coder-Next

#270

Post by theshrike79 »

cyanydeez wrote: post #250
theshrike79 wrote: post #36

Earlier quoted context omitted.

Mode like open local models are becoming "good enough". I got stuff done with Sonnet 3.7 just fine, it did need a bunch of babysitting, but still it was a net positive to productivity. Now local models are at that level, closing up on the current SOTA. When "anyone" can run an Opus 4.5 level model at home, we're going to be getting diminishing returns from closed online-only models.

See, the market is investing like _that will never happen_.

I'm just riding the VC powered wave of way-too-cheap online AI services and building tools and scaffolding to prepare for the eventual switch to local models =)
Post reply on HN