Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-Coder-Next

qwen.ai

Post reply on HN
Print view View original

351–360 of 443 posts

Re: Qwen3-Coder-Next

#351

Post by kristianp »

simonw wrote: post #3

This GGUF is 48.4GB - https://huggingface.co/Qwen/Qwen3-Coder-Next-GGUF/tree/main/... - which should be usable on higher end laptops. I still haven't experienced a local model that fits on my 64GB MacBook Pro and can run a coding agent like Codex CLI or Claude code well enough to be useful. Maybe this will be the one? This Unsloth guide from a sibling comment suggests it might be: https://unsloth.ai/docs/models/qwen3…

Why don't you try it out in Opencode? It's possible to hook up the openrouter api, and some providers have started to host it there [1]. It's not yet available in opencode's model list [2].

Opencode's /connect command has a big list of providers, openrouter is on there.

[1] https://openrouter.ai/qwen/qwen3-coder-next

[2] https://opencode.ai/docs/zen/#endpoints

Re: Qwen3-Coder-Next

#352

Post by justaboutanyone »

cmrdporcupine wrote: post #348
adefa wrote: post #314

Benchmarks using DGX Spark on vLLM 0.15.1.dev0+gf17644344 FP8: https://huggingface.co/Qwen/Qwen3-Coder-Next-FP8 Sequential (single request) Prompt Gen Prompt Processing Token Gen Tokens Tokens (tokens/sec) (tokens/sec) ------ ------ ----------------- ----------- 521 49 3,157 44.2 1,033 83 3,917 43.7 2,057 77 3,937 43.6 4,105 77 4,453 43.2 8,201 77 4,710 42.2 Parallel (concurrent requests) pp4096+tg128 (4K context, 12…

I tried the FP8 in vLLM on my Spark and although it fit in memory, I started swapping once I actually tried to run any queries, and, yeah, could not have a context larger than 8k. I figured out later this is because vLLM apparently de-quantizes to BF16 at runtime, so pointless to run the FP8? I get about 30-35 tok/second using llama.cpp and a 4-bit quant. And a 200+k context, using only 50GB of RAM.

Running llama.cpp rather than vLLM, it's happy enough to run the FP8 variant with 200k+ context using about 90GB vram

Re: Qwen3-Coder-Next

#353

Post by danielhanchen »

CamperBob2 wrote: post #267
danielhanchen wrote: post #2

For those interested, made some Dynamic Unsloth GGUFs for local deployment at https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF and made a guide on using Claude Code / Codex locally: https://unsloth.ai/docs/models/qwen3-coder-next

Good results with your Q8_0 version on 96GB RTX 6000 Blackwell. It one-shotted the Flappy Bird game and also wrote a good Wordle clone in four shots, all at over 60 tps. Thanks! Is your Q8_0 file the same as the one hosted directly on the Qwen GGUF page?

Nice! Yes Q8_0 is similar - the others are different since they use a calibration dataset.

Re: Qwen3-Coder-Next

#354

Post by danielhanchen »

ranger_danger wrote: post #326
danielhanchen wrote: post #14

Earlier quoted context omitted.

UD stands for "Unsloth-Dynamic" which upcasts important layers to higher bits. Non UD is just standard llama.cpp quants. Both still use our calibration dataset.

What is your definition of "important" in this context?

Oh we wrote about it here: https://unsloth.ai/docs/basics/unsloth-dynamic-2.0-ggufs

Re: Qwen3-Coder-Next

#355

Post by CryptoBanker »

falloutx wrote: post #336
Aurornis wrote: post #197

Earlier quoted context omitted.

> Then Anthropic started saying I’m not allowed to use my Claude Code subscription with my preferred tools To be clear, since this confuses a lot of people in every thread: Anthropic will let you use their API with any coding tools you want. You just have to go through the public API and pay the same rate as everyone else. They have not "blocked" or "banned" any coding tools from using their API, even though a lot of…

> Anthropic will let you use their API with any coding tools you want No, in 2026, even with their API plan the create key is disabled for most orgs, you basically have to ask your admin to give you a key to use something other than Claude Code. You can imagine how that would be a problem.

That’s not an Anthropic problem, that’s a problem with whomever you work for.

Re: Qwen3-Coder-Next

#356

Post by danielhanchen »

Keats wrote: post #265
mirekrusin wrote: post #251

Earlier quoted context omitted.

It takes download time + 1 minute to test speed yourself, you can try different quants, it's hard to write down a table because it depends on your system ie. ram clock etc. if you go out of gpu. I guess it would make sense to have something like max context size/quants that fit fully on common configs with gpus, dual gpus, unified ram on mac etc.

Testing speed is easy yes, I'm mostly wondering about the quality difference between Q6 vs Q8_K_XL for example.

I haven't done benchmarking yet (plan to do them), but it should be similar to our post on DeepSeek-V3.1 Dynamic GGUFs: https://unsloth.ai/docs/basics/unsloth-dynamic-2.0-ggufs

Re: Qwen3-Coder-Next

#357

Post by danielhanchen »

bityard wrote: post #190
danielhanchen wrote: post #2

For those interested, made some Dynamic Unsloth GGUFs for local deployment at https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF and made a guide on using Claude Code / Codex locally: https://unsloth.ai/docs/models/qwen3-coder-next

Hi Daniel, I've been using some of your models on my Framework Desktop at home. Thanks for all that you do. Asking from a place of pure ignorance here, because I don't see the answer on HF or in your docs: Why would I (or anyone) want to run this instead of Qwen3's own GGUFs?

Thanks! Oh Qwen3's own GGUFs also works, but ours are dynamically quantized and calibrated with a reasonably large diverse dataset, whilst Qwen's ones are not - see https://unsloth.ai/docs/basics/unsloth-dynamic-2.0-ggufs

Re: Qwen3-Coder-Next

#358

Post by danielhanchen »

genpfault wrote: post #139
danielhanchen wrote: post #2

For those interested, made some Dynamic Unsloth GGUFs for local deployment at https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF and made a guide on using Claude Code / Codex locally: https://unsloth.ai/docs/models/qwen3-coder-next

Nice! Getting ~39 tok/s @ ~60% GPU util. (~170W out of 303W per nvtop). System info: $ ./llama-server --version ggml_vulkan: Found 1 Vulkan devices: ggml_vulkan: 0 = Radeon RX 7900 XTX (RADV NAVI31) (radv) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat version: 7897 (3dd95914d) built with GNU 11.4.0 for Linux x86_64 llama.cpp command-line: $ ./llama-server…

Super cool! Also with `--fit on` you don't need `--ctx-size 32768` technically anymore - llama-server will auto determine the max context size!

Re: Qwen3-Coder-Next

#359

Post by hedora »

christkv wrote: post #292
kristopolous wrote: post #255

Earlier quoted context omitted.

We need a new word, not "local model" but "my own computers model" CapEx based This distinction is important because some "we support local model" tools have things like ollama orchestration or use the llama.cpp libraries to connect to models on the same physical machine. That's not my definition of local. Mine is "local network". so call it the "LAN model" until we come up with something better. "Self-host" exists b…

I won't need a heater with that running in my room.

This looks like it’ll run easily on a Strix Halo (180W TDP), and be a little sluggish on previous gen AMDs (80W TDP).

I can’t be bothered to check TDPs on 64GB macbooks, but none of these devices really count as space heaters.

Re: Qwen3-Coder-Next

#360

Post by danielhanchen »

simonw wrote: post #335
technotony wrote: post #263

Earlier quoted context omitted.

what are your impressions?

I got Codex CLI running against it and was sadly very unimpressed - it got stuck in a loop running "ls" for some reason when I asked it to create a new file.

Yes sadly that sometimes happens - the issue is Codex CLI / Claude Code were designed for GPT / Claude models specifically, so it'll be hard for OSS models directly to utilize the full spec / tools etc, and might get loops sometimes - I would maybe try the MXFP4_MOE quant to see if it helps, and maybe try Qwen CLI (was planning to make a guide for it as well)

I guess until we see the day OSS models truly utilize Codex / CC very well, then local models will really take off

Post reply on HN