Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model • Hacker News Board

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#191

Post by cpburns2009 » Wed, Apr 22, 2026, 6:09 PM UTC

genpfault wrote: post #59

spwa4 wrote: post #15

Unsloth quants available: https://unsloth.ai/docs/models/qwen3.6

Getting ~36-33 tok/s (see the "S_TG t/s" column) on a 24GB Radeon RX 7900 XTX using llama.cpp's Vulkan backend: $ llama-server --version version: 8851 (e365e658f) $ llama-batched-bench -hf unsloth/Qwen3.6-27B-GGUF:IQ4_XS -npp 1000,2000,4000,8000,16000,32000 -ntg 128 -npl 1 -c 34000 | PP | TG | B | N_KV | T_PP s | S_PP t/s | T_TG s | S_TG t/s | T s | S t/s | |-------|--------|------|--------|----------|----------|----…

~25-26 tok/s with ROCm using the same card, llama.cpp b8884:

    $ llama-batched-bench -dev ROCm1 -hf unsloth/Qwen3.6-27B-GGUF:IQ4_XS -npp 1000,2000,4000,8000,16000,32000 -ntg 128 -npl 1 -c 34000
    |    PP |     TG |    B |   N_KV |   T_PP s | S_PP t/s |   T_TG s | S_TG t/s |      T s |    S t/s |
    |-------|--------|------|--------|----------|----------|----------|----------|----------|----------|
    |  1000 |    128 |    1 |   1128 |    1.034 |   966.90 |    4.851 |    26.39 |    5.885 |   191.67 |
    |  2000 |    128 |    1 |   2128 |    2.104 |   950.38 |    4.853 |    26.38 |    6.957 |   305.86 |
    |  4000 |    128 |    1 |   4128 |    4.269 |   937.00 |    4.876 |    26.25 |    9.145 |   451.40 |
    |  8000 |    128 |    1 |   8128 |    8.962 |   892.69 |    4.912 |    26.06 |   13.873 |   585.88 |
    | 16000 |    128 |    1 |  16128 |   19.673 |   813.31 |    4.996 |    25.62 |   24.669 |   653.78 |
    | 32000 |    128 |    1 |  32128 |   46.304 |   691.09 |    5.122 |    24.99 |   51.426 |   624.75 |

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#192

Post by cyanydeez » Wed, Apr 22, 2026, 6:10 PM UTC

danielhanchen wrote: post #73

ryandrake wrote: post #67

Earlier quoted context omitted.

Yea, this is currently the confusing part of running local models for newbies: Even after you have decided which model you want to run, and which org's quantizations to use (let's just assume Unsloth's for example), there are often dozens of quantizations offered, and choosing among them is confusing. Say you have a GPU with 20GB of VRAM. You're probably going to be able to run all the 3-bit quantizations with no pro…

We made Unsloth Studio which should help :) 1. Auto best official parameters set for all models 2. Auto determines the largest quant that can fit on your PC / Mac etc 3. Auto determines max context length 4. Auto heals tool calls, provides python & bash + web search :)

Is unsloth working on managing remote servers, like how vscode integrates with a remote server via ssh?

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#193

Post by zargon » Wed, Apr 22, 2026, 6:10 PM UTC

kpw94 wrote: post #160

zargon wrote: post #144

Earlier quoted context omitted.

If someone doesn't specifically say prefill then they always mean decode speed. I have never seen an exception. Most people just ignore prefill.

But isn't the prefill speed the bottleneck in some systems* ? Sure it's order of magnitude faster (10x on Apple Metal?) but there's also order of magnitude more tokens to process, especially for tasks involving summarization of some sort. But point taken that the parent numbers are probably decode * Specifically, Mac metal, which is what parent numbers are about

Yes, definitely it's the bottleneck for most use cases besides "chatting". It's the reason I have never bought a Mac for LLM purposes.

It's frustrating when trying to find benchmarks because almost everyone gives decode speed without mentioning prefill speed.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#194

Post by jedisct1 » Wed, Apr 22, 2026, 6:11 PM UTC

I really like local models for code reviews / security audits.

Even if they don't run super fast, I can let them work overnight and get comprehensive reports in the morning.

I used Qwen3.6-27B on an M5 (oq8, using omlx) and Swival (https://swival.dev) /audit command on small code bases I use for benchmarking models for security audits.

It found 8 out of 10, which is excellent for a local model, produced valid patches, and didn't report any false positives. which is even better.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#195

Post by simonw » Wed, Apr 22, 2026, 6:12 PM UTC

rob wrote: post #177

refulgentis wrote: post #155

Earlier quoted context omitted.

Because I want to read about Qwen, not someone's one-off vibe test followed by 1:1 conversations. (case in miniature here: which is the last comment in this thread that says something about Qwen? The root post. Is that fun policing? Yes, apologies.)

I think it's to help drive traffic to his blog now that he's accepted sponsors in the header of every page. I do see this pelican thing come up from him on every model post that gets released.

The traffic I get from a comment with a link to a pelican is pretty tiny.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#196

Post by anonym29 » Wed, Apr 22, 2026, 6:13 PM UTC

adrian_b wrote: post #190

anonym29 wrote: post #139

Earlier quoted context omitted.

I use llama.cpp, and there is a way to do this - some layers to (i)GPU, the rest to CPU. I was just trying this out with Kimi K2.5 (in preparation for trying it out with Kimi K2.6 the other night. Check out the --n-cpu-moe flag in llama.cpp. That said, my Strix Halo rig only has PCIe 4.0 for my NVMe, and I'm using a 990 Evo that had poor sustained random read, being DRAM-less. My effective read speeds from disk were…

In a computer with 2 PCIe 5.0 SSDs or one with a PCIe 5.0 SSDs and a PCIe 4.0 SSD, it should be possible to stream weights from the SSDs at 20 GB/s, or even more. This is not a little faster, but 10 times faster than on your system. So a couple of tokens per second generation speed should be achievable. Nowadays even many NUCs or NUC-like mini-PCs have such SSD slots. I have actually started working at optimizing suc…

Strix Halo, to my knowledge, does not support PCIe 5.0 NVMe drives, unfortunately, despite it being Zen 5, and Zen 5 supporting the PCIe 5.0 standard.

While many other NUCs may support them, what most of them lack compared to Strix Halo is a 128 GB pool of unified LPDDR5x-8000 on a 256 bit bus and the Radeon 8060S iGPU with 40 CU of RDNA 3.5, which is roughly equivalent in processing power to a laptop 4060 or desktop 3060.

The Radeon 780M and Radeon 890M integrated graphics that come on most AMD NUCs don't hold a candle to Strix Halo's 8060S, and what little you'd gain in this narrow use case with PCIe gen 5, you'd lose a lot in the more common use cases of models that can fit into a 128 GB pool of unified memory, and there are some really nice ones.

Also, the speeds you're suggesting seem rather optimistic. Gen 5 drives, as I understand, hit peak speeds of about 28-30 GB/s (with two in RAID0, at 14-15 GB/s each), but that's peak sequential reads, which is neither reflective of sustained reads, nor the random read workloads that dominate reading model weights.

Maybe there are some Intel NUCs that compete in this space that I'm less up to speed with which do support PCIe 5. I know Panther Lake costs about as much to manufacture as Strix Halo, and while it's much more power efficient and achieves a lot more compute per Xe3 graphics core than Strix Halo achieves per RDNA 3.5 CU, they Panther Lake that's actually shipping ships with so many fewer Xe3 cores that it's still a weaker system overall.

Maybe DGX Spark supports PCIe 5.0, I don't own one and am admittedly not as familiar with that platform either, though it's worth mentioning that the price gap between Strix Halo and DGX Spark at launch ($2000 vs $4000) has closed a bit (many Strix Halo run $3000 now, vs $4700 for DGX Spark, and I think some non-Nvidia GB10 systems are a bit cheaper still)

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#197

Post by fnordpiglet » Wed, Apr 22, 2026, 6:13 PM UTC

otabdeveloper4 wrote: post #125

fnordpiglet wrote: post #84

Earlier quoted context omitted.

For coding often quality at the margin is crucial even at a premium. It’s not the same as cranking out spam emails or HN posts at scale. This is why the marginal difference between your median engineer and your P99 engineer is comp is substantial, while the marginal comp difference between your median pick and packer vs your P99 pick and packer isn’t. I’d also say it keeps the frontier shops competitive while costing…

> For coding often quality at the margin is crucial even at a premium. For coding, quality is not measurable and is based entirely on feels (er, sorry, "vibes"). Employers paying for SOTA models is nothing but a lifestyle status perk for employees, like ping-pong tables or fancy lunch snacks.

I’m building my own company and I consider model choice crucial to my marginal ability to produce a higher quality product I don’t regret having built. Every higher end dev shop I’ve worked at over the last few years perceives things the same. There are measurable outcomes from software built well and software not, even if the code itself isn’t easily measurable. I would rather pay a few thousand more per year for a better overall outcome with less developer struggle against bad model decisions than end up with an inferior end product and have expensive developer spin wheels containing a dumb as a brick model. But everyone’s career experiences are different and I’d feel sad to work at a place where SOTA is a lifestyle choice rather than a rational engineering and business choice.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#198

Post by fnordpiglet » Wed, Apr 22, 2026, 6:16 PM UTC

datadrivenangel wrote: post #173

AJ007 wrote: post #90

Earlier quoted context omitted.

Not sure how your last point matters if 27b can run on consumer hardware, besides being hosted by any company which the user could certainly trust more than anthropic. OpenAI & Anthropic are just lying to everyone right now because if they can't raise enough money they are dead. Intelligence is a commodity, the semiconductor supply chain is not.

The challenge is token speed. I did some local coding yesterday with qwen3.6 35b and getting 10-40 tokens per second means that the wall time is much longer. 20 tokens per second is a bit over a thousand tokens per minute, which is slower than the the experience you get with Claude Code or the opus models. Slower and worse is still useful, but not as good in two important dimensions.

Also benchmark measures are not empirical experience measures and are well gamed. As other commenters have said the actual observed behavior is inferior, so it’s not just speed.

It’s ludicrous to believe a small parameter count model will out perform a well made high parameter count model. That’s just magical thinking. We’ve not empirically observed any flattening of the scaling laws, and there’s no reason to believe the scrappy and smart qwen team has discovered P=NP, FTL, or the magical non linear parameter count scaling model.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#199

Post by cyanydeez » Wed, Apr 22, 2026, 6:16 PM UTC

benob wrote: post #27

anonzzzies wrote: post #11

I wish that all announcements of models would show what (consumer) hardware you can run this on today, costs and tok/s.

I get ~5 tokens/s on an M4 with 32G of RAM, using: llama-server \ -hf unsloth/Qwen3.6-27B-GGUF:Q4_K_M \ --no-mmproj \ --fit on \ -np 1 \ -c 65536 \ --cache-ram 4096 -ctxcp 2 \ --jinja \ --temp 0.6 \ --top-p 0.95 \ --top-k 20 \ --min-p 0.0 \ --presence-penalty 0.0 \ --repeat-penalty 1.0 \ --reasoning on \ --chat-template-kwargs '{"preserve_thinking": true}' 35B-A3B model is at ~25 t/s. For comparison, on an A100 (~RTX…

Using opencode and Qwen-Coder-Next I get it reliably up to about 85k before it takes too long to respond.

I tried the other qwen models and the reasoning stuff seems to do more harm than good.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#200

Post by refulgentis » Wed, Apr 22, 2026, 6:16 PM UTC

simonw wrote: post #186

refulgentis wrote: post #155

Earlier quoted context omitted.

Because I want to read about Qwen, not someone's one-off vibe test followed by 1:1 conversations. (case in miniature here: which is the last comment in this thread that says something about Qwen? The root post. Is that fun policing? Yes, apologies.)

There's a bunch of useful information in my comment that's independent of the fact that it drew a pelican: 1. You can run this on a Mac using llama-server and a 17GB downloaded file 2. That version does indeed produce output (for one specific task) that's of a good enough quality to be worth spending more time checking out this model 3. It generated 4,444 tokens in 2min 53s, which is 25.57 tokens/s

Right, that is exactly what I meant by "the root post [had info about Qwen]" - you shouldn't feel I'm being critical of you or asking you to do anything different, at all. I admire you deeply and feel humbled* by interacting with you, so I really want that to be 100% clear, because this is the 2nd time I'm reading that it might be personal.

* er, that probably sounds strange, but I did just spend 6 weeks working on integrating the Willison Trifecta for my app I've been building for 2.5 years, and I considered it a release blocker. It's a simple mental model that is a significant UX accomplishment IMHO.