Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

qwen.ai

Post reply on HN
Print view View original

361–370 of 482 posts

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#361

Post by girvo »

pawelduda wrote: post #297
finnjohnsen2 wrote: post #261

Since Gemma 4 came this easter the gap from self hosting models to Claude has decreased sigificantly I think. The gap is still huge it just that local models were extremely non-competitive before easter. So now it seems Qwen 3.6 is another bump up from Gemma 4 which is exciting if it is so. I keep an Opus close ofcourse, because these local models still wander off in the wrong direction and fails. Something Opus almo…

How many tokens/s do you get on RTX 5090?

Not that you asked but I’m getting ~20 tokens/s on my DGX Spark (Asus actually) using an Int4 AutoRound quant, MTP 1 and some other tricks

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#363

Post by 2001zhaozhao »

2001zhaozhao wrote: post #202

I'm kind of interested in a setup where one buys local hardware specifically to run a crap ton of small-to-medium LLM locally 24/7 at high throughput. These models might now be smart enough to make all kinds of autonomous agent workflows viable at a cheap price, with a good queue prioritization system for queries to fully utilize the hardware.

Adding to my own comment now that I've read the announcement in a little more detail: I find the assertion that the model's coding performance surpasses their own flagship 397B model from last generation fairly convincing.

This sounds like significant genuine gains unless one of the following is true, which would be really unlikely:

1. They somehow managed to benchmaxx every coding benchmark way harder than their own last generation.

2. They held back the coding performance of their last generation 397B model on purpose to make this 3.6 Qwen model look good. (basically a tinfoil hat theory as it would literally require 4D chess and self-harming to do)

So, it's pretty save to say that we actually have a competent agentic coding model we can leave on in a prosumer laptop overnight to create real software for almost zero token costs.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#364

Post by ItsClo688 »

simonw wrote: post #106

The pelican is excellent for a 16.8GB quantized local model: https://simonwillison.net/2026/Apr/22/qwen36-27b/ I ran it on an M5 Pro with 128GB of RAM, but it only needs ~20GB of that. I expect it will run OK on a 32GB machine. Performance numbers: Reading: 20 tokens, 0.4s, 54.32 tokens/s Generation: 4,444 tokens, 2min 53s, 25.57 tokens/s I like it better than the pelican I got from Opus 4.7 the other day: https://si…

[flagged]

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#365

Post by RandyOrion »

Thank you Qwen team. Small DENSE LLMs shapes the future of local LLM users.

When Qwen 3.5 27b released, I didn't really understand why linear attention is used instead of full attention because of the performance degradation and problems introduced with extra (linear) operators. After doing some tests, I found that with llama.cpp and IQ4_XS quant, the model and BF16 cache of the whole 262k context just fit on 32GB vram, which is impossible with full attention. In contrast, with gemma 4 31b IQ4_XS quant I have to use Q8_0 cache to fit 262k context on the vram, which is a little annoying (no offenses, thank you gemma team, too).

From benchmarks, 3.5->3.6 upgrade is about agent things. I hope future upgrades fix some problems I found, e.g., output repetitiveness in long conversations and knowledge broadness.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#366

Post by 2001zhaozhao »

kroaton wrote: post #253
2001zhaozhao wrote: post #202

I'm kind of interested in a setup where one buys local hardware specifically to run a crap ton of small-to-medium LLM locally 24/7 at high throughput. These models might now be smart enough to make all kinds of autonomous agent workflows viable at a cheap price, with a good queue prioritization system for queries to fully utilize the hardware.

Buy any Strix Halo box and have fun with your 128GB of VRAM.

I wonder whether it is much more cost-effective in terms of token throughput / hardware+power cost to get actual GPUs instead, given that the model size is only 27B.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#367

Post by tgtweak »

ItsClo688 wrote: post #364
simonw wrote: post #106

The pelican is excellent for a 16.8GB quantized local model: https://simonwillison.net/2026/Apr/22/qwen36-27b/ I ran it on an M5 Pro with 128GB of RAM, but it only needs ~20GB of that. I expect it will run OK on a 32GB machine. Performance numbers: Reading: 20 tokens, 0.4s, 54.32 tokens/s Generation: 4,444 tokens, 2min 53s, 25.57 tokens/s I like it better than the pelican I got from Opus 4.7 the other day: https://si…

[flagged]

I've been using it in a few harnesses (FP8 quant, max context length) and it does seem to get tripped up by tool use, often repeating the same tool when it failed previously - that's usually not a great sign for long-term context and multi-step reasoning. It is excellent at one-shotting though and might be most useful as a sub-agent for a stronger frontier coordinator.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#368

Post by 2001zhaozhao »

OliverGuy wrote: post #278
2001zhaozhao wrote: post #202

I'm kind of interested in a setup where one buys local hardware specifically to run a crap ton of small-to-medium LLM locally 24/7 at high throughput. These models might now be smart enough to make all kinds of autonomous agent workflows viable at a cheap price, with a good queue prioritization system for queries to fully utilize the hardware.

I would love to have a shit load of small (27B dense. 35B MoE) agents running locally and looking at and ingesting every bit of data about me, my life and what I get up to see what sort of correlations it finds. Give a coding agent access to a data lake of events and let it build up its own analytics tooling to extract and draw out information from that data, and present it to me as daily/weekly/monthly summaries.

This was along my lines of thinking at one point as well. Though I'm now more interested in having it experiment autonomously on my software projects overnight.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#369

Post by 2001zhaozhao »

Havoc wrote: post #323
OliverGuy wrote: post #278

Earlier quoted context omitted.

I would love to have a shit load of small (27B dense. 35B MoE) agents running locally and looking at and ingesting every bit of data about me, my life and what I get up to see what sort of correlations it finds. Give a coding agent access to a data lake of events and let it build up its own analytics tooling to extract and draw out information from that data, and present it to me as daily/weekly/monthly summaries.

That's definitely doable. Planning similar except more webscraping / newsfeed / monitoring like. I've got 3x SBCs that can run the Gemma 4 26B MoE on NPU. Around 4W extra power, 3 tokens a second...so that can hammer away at tasks 24/7 without moving the needle on electricity bill

I wonder if some investment firms are already doing this internally at a large scale. (Probably.)

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#370

Post by SV_BubbleTime »

SlavikCA wrote: post #237
maxloh wrote: post #230

Earlier quoted context omitted.

I thought Q4_K_M is the standard. Why did you choose the 6-bit variant? Does it generate better input?

There is no standard. The higher quantization - the better results, but more memory is needed. Q8 is the best.

FP32 is best, although I wonder if there isn’t something better I don’t know about. Q8 is for the most part equal to FP16 in practical terms by being smart about what is quantized, but iirc always slower than FP16 and FP8.
Post reply on HN