Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

qwen.ai

Post reply on HN
Print view View original

121–130 of 482 posts

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#121

Post by NitpickLawyer »

esafak wrote: post #7
amunozo wrote: post #2

A bit skeptical about a 27B model comparable to opus...

Some of these benchmarks are supposedly easy to game. Which ones should we pay attention to?

SWE-REbench should not be gameable. They collect new issues from live repos, and if you check 1-2 months after a model was released, you can get an idea. But even that would be "benchmaxxxable", which is an overloaded term that can mean many things, but the most vanilla interpretation is that with RL you can get a model to follow a certain task pretty well, but it'll get "stuck" on that task type, or "stubborn" when asked similar but sufficiently different tasks. So for swe-rebench that would be "it fixes bugs in these types of repos, under this harness, but ask it to do soemthing else in a repo and you might not get the same results". In a nutshell.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#122

Post by acidtechno303 »

syntaxing wrote: post #97
Aurornis wrote: post #75

Earlier quoted context omitted.

I use Opus and the Qwen models. The gap between them is much larger than the benchmark charts show. If you want to compare to a hosted model, look toward the GLM hosted model. It’s closest to the big players right now. They were selling it at very low prices but have started raising the price recently.

Yes and no. Are you using open router or local? Are the models are good as Opus? No. But 99% of the time, local models are terrible because of user errors. Especially true for MoE, even though the perplexity only drops minimal for Q4 and q4_0 for the KV cache, the models get noticeably worse.

Sounds like you're accusing a professional of holding their tool incorrectly. Not impossible, but not likely either.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#123

Post by simonw »

throwaw12 wrote: post #108
simonw wrote: post #106

The pelican is excellent for a 16.8GB quantized local model: https://simonwillison.net/2026/Apr/22/qwen36-27b/ I ran it on an M5 Pro with 128GB of RAM, but it only needs ~20GB of that. I expect it will run OK on a 32GB machine. Performance numbers: Reading: 20 tokens, 0.4s, 54.32 tokens/s Generation: 4,444 tokens, 2min 53s, 25.57 tokens/s I like it better than the pelican I got from Opus 4.7 the other day: https://si…

I feel like this time it is indeed in the training set, because it is too good to be true. Can you run your other tests and see the difference?

It went pretty wild with "Generate an SVG of a NORTH VIRGINIA OPOSSUM ON AN E-SCOOTER":

https://gist.github.com/simonw/95735fe5e76e6fdf1753e6dcce360...

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#124

Post by CamperBob2 »

anonzzzies wrote: post #11

I wish that all announcements of models would show what (consumer) hardware you can run this on today, costs and tok/s.

27B dense is not a screamer, even on an RTX 6000, but it will run at full precision with (more than) enough room for context at the model's own capacity. You can expect about 30 tokens/second after prompt processing. Quants will likely run similarly well on the 16/24/32 GB consumer GPUs.

The 3.5 27B model was a strong and capable reasoner, so I have high hopes for this one. Thanks to the team at Qwen for keeping competition in this space alive.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#125

Post by otabdeveloper4 »

fnordpiglet wrote: post #84
jameson wrote: post #74

What competitive advantage does OpenAI/Anthropic has when companies like Qwen/Minimax/etc are open sourcing models that shows similar (yet below than OpenAI/Anthropic) benchmark results? Also, the token prices of these open source models are at a fraction of Anthropic's Opus 4.6[1] [1]: https://artificialanalysis.ai/models/#pricing

For coding often quality at the margin is crucial even at a premium. It’s not the same as cranking out spam emails or HN posts at scale. This is why the marginal difference between your median engineer and your P99 engineer is comp is substantial, while the marginal comp difference between your median pick and packer vs your P99 pick and packer isn’t. I’d also say it keeps the frontier shops competitive while costing…

> For coding often quality at the margin is crucial even at a premium.

For coding, quality is not measurable and is based entirely on feels (er, sorry, "vibes").

Employers paying for SOTA models is nothing but a lifestyle status perk for employees, like ping-pong tables or fancy lunch snacks.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#126

Post by zozbot234 »

anonym29 wrote: post #117
Aurornis wrote: post #23

Earlier quoted context omitted.

The 27B model they release directly would require significant hardware to run natively at 16-bit: A Mac or Strix Halo 128GB system, multiple high memory consumer GPUs, or an RTX 6000 workstation card. This is why they don’t advertise which consumer hardware it can run on: Their direct release that delivers these results cannot fit on your average consumer system. Most consumers don’t run the model they release direct…

You absolutely do not need to run at full BF16. The quality loss between BF16 (55.65 GB in GGUF) and Q8_0 (30.44 GB in GGUF) is essentially zero - think on the order of magnitude of +0.01-0.03 perplexity, or ~0.1-0.3% relative PPL increase. The quality loss between BF16 and Q4_K_M (18.66 GB in GGUF) is close to imperceptible, with perplexity changes in the +0.1-0.3 ballpark, or ~1-3% relative PPL increase. This would…

I'm all for running large MoE models on unified memory systems, but developers of inference engines should do a better job of figuring out how to run larger-than-total-RAM models on such systems, streaming in sparse weights from SSD but leveraging the large unified memory as cache. This is easily supported with pure-CPU inference via mmap, but there is no obvious equivalent when using the GPU for inference.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#127

Post by NitpickLawyer »

lreeves wrote: post #78
alfonsodev wrote: post #13

Earlier quoted context omitted.

good to hear! Do you mind sharing your setup and tokens / seconds performance ?

I'm running the unquantized base model on 2xA6000s (Ampere gen, 48GB each). Runs at about 25 tokens/second.

FYI they also released FP8 quants, and those should be faster on your setup (we have the same). As long as you keep kv at 16bit, FP8 should be close-to-lossless compared to 16bit, but with more context available and faster inference speed.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#128

Post by johndough »

genpfault wrote: post #59
spwa4 wrote: post #15

Unsloth quants available: https://unsloth.ai/docs/models/qwen3.6

Getting ~36-33 tok/s (see the "S_TG t/s" column) on a 24GB Radeon RX 7900 XTX using llama.cpp's Vulkan backend: $ llama-server --version version: 8851 (e365e658f) $ llama-batched-bench -hf unsloth/Qwen3.6-27B-GGUF:IQ4_XS -npp 1000,2000,4000,8000,16000,32000 -ntg 128 -npl 1 -c 34000 | PP | TG | B | N_KV | T_PP s | S_PP t/s | T_TG s | S_TG t/s | T s | S t/s | |-------|--------|------|--------|----------|----------|----…

Getting ~44-40 tok/s on 24GB RTX 3090 (llama.cpp version 8884, same llama-batched-bench call):

    |    PP |     TG |    B |   N_KV |   T_PP s | S_PP t/s |   T_TG s | S_TG t/s |      T s |    S t/s |
    |-------|--------|------|--------|----------|----------|----------|----------|----------|----------|
    |  1000 |    128 |    1 |   1128 |    0.684 |  1462.61 |    2.869 |    44.61 |    3.553 |   317.47 |
    |  2000 |    128 |    1 |   2128 |    1.390 |  1438.84 |    2.868 |    44.64 |    4.258 |   499.80 |
    |  4000 |    128 |    1 |   4128 |    2.791 |  1433.18 |    2.886 |    44.35 |    5.677 |   727.11 |
    |  8000 |    128 |    1 |   8128 |    5.646 |  1416.98 |    2.922 |    43.80 |    8.568 |   948.65 |
    | 16000 |    128 |    1 |  16128 |   11.851 |  1350.10 |    3.007 |    42.57 |   14.857 |  1085.51 |
    | 32000 |    128 |    1 |  32128 |   25.855 |  1237.66 |    3.168 |    40.40 |   29.024 |  1106.96 |
Edit: Model gets stuck in infinite loops at this quantization level. I've also tried Q5_K_M quantization (fits up to 51968 context length), which seems more robust.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#129

Post by otabdeveloper4 »

cbg0 wrote: post #72
originalvichy wrote: post #8

Earlier quoted context omitted.

For at least a year now, it has been clear that data quality and fine-tuning are the main sources of improvement for mediym-level models. Size != quality for specialized, narrow use cases such as coding. It’s not a surprise that models are leapfrogging each other when the engineers are able to incorporate better code examples and reasoning traces, which in turn bring higher quality outputs.

If all you're looking at is benchmarks that might be true, but those are way too easy to game. Try using this model alongside Opus for some work in Rust/C++ and it'll be night and day. You really can't compare a model that's got trillions of parameters to a 27B one.

> ...and it'll be night and day.

That's just, like, your opinion, man.

> You really can't compare a model that's got trillions of parameters to a 27B one.

Parameter count doesn't matter much when coding. You don't need in-depth general knowledge or multilingual support in a coding model.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#130

Post by throwaw12 »

simonw wrote: post #123
throwaw12 wrote: post #108

Earlier quoted context omitted.

I feel like this time it is indeed in the training set, because it is too good to be true. Can you run your other tests and see the difference?

It went pretty wild with "Generate an SVG of a NORTH VIRGINIA OPOSSUM ON AN E-SCOOTER": https://gist.github.com/simonw/95735fe5e76e6fdf1753e6dcce360...

compared to your test with GLM 5.1, this indeed looks off

https://xcancel.com/simonw/status/2041646779553476801

Post reply on HN