Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

qwen.ai

Post reply on HN
Print view View original

291–300 of 482 posts

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#291

Post by kombine »

acrispino wrote: post #183
ndriscoll wrote: post #70

Earlier quoted context omitted.

Note that you could also run them on AMD (and presumably Intel) dGPUs. e.g. I have a 32GB R9700, which is much cheaper than a 5090, and runs 27B dense models at ~20 t/s (or MoE models with 3-4B active at ~80t/s). I expect an Arc B70 would also work soon if it doesn't already, and would likely be the price/perf sweet spot right now. My R9700 does seem to have an annoying firmware or driver bug[0] that causes the fan t…

I have 2x asrock R9700. One of the them was noticeably noisier than the other and eventually developed an annoying vibration while in the middle of its fan curve. Asrock replaced it under RMA.

How is your experience with dual cards? Is the a dense 27B model the best what you can run on this setup? What about other applications eg. diffusion or fine-tuning?

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#292

Post by noman-land »

datadrivenangel wrote: post #279

So far I'm unimpressed for local inference. got 11 tokens per second on omlx on an M5 Pro with 128gb of ram, so it took an hour to write a few hundred lines of code that didn't work. Opus and Sonnet in CC the same task successfully in a matter of minutes. The 3.6:35b model seemed okay on ollama yesterday. Need to check out other harnesses for this besides claude code, but the local models are just painfully slow.

OpenCode seems to be a lot better than Claude at using local models.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#293

Post by carlovalenti »

alex7o wrote: post #185
Aurornis wrote: post #43

Earlier quoted context omitted.

> As this is a dense model and it's pretty sizable, 4-bit quantization can be nearly lossless The 4-bit quants are far from lossless. The effects show up more on longer context problems. > You can probably even go FP8 with 5090 (though there will be tradeoffs) You cannot run these models at 8-bit on a 32GB card because you need space for context. Typically it would be Q5 on a 32GB card to fit context lengths needed f…

Turboquant on 4bit helps a lot as well for keeping context in vram, but int4 is definitely not lossless. But it all depends for some people this is sufficient

[dead]

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#294

Post by regularfry »

Aurornis wrote: post #35
GaggiX wrote: post #20

Earlier quoted context omitted.

At 4-bit quantization it should already fit quite nicely.

Unfortunately not with a reasonable context length.

I've got 139k context with the UD-Q4_K_XL on a 4090, q8_0 ctk/v. Could probably squeeze a little more but that's enough for me for the moment.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#295

Post by mudkipdev »

mft_ wrote: post #271

Huh, running the Q4_K_M quant with LM Studio, and asked it "How can I set up Qwen 3.6 27b to use tools and access the local file system?". Part of its reply was: Quick clarification: As of early 2025, "Qwen 3.6" hasn't been released yet. You are likely looking for Qwen2.5, specifically the Qwen2.5-32B-Instruct model, which is the 30B-class model closest to your 27B reference. The instructions below will use this mode…

If you are talking with Claude about AI, it will sometimes passively bring up "frontier models like GPT-4o"

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#296

Post by holoduke »

bityard wrote: post #76
anonzzzies wrote: post #11

I wish that all announcements of models would show what (consumer) hardware you can run this on today, costs and tok/s.

There are infinite combinations of CPU/GPU capable of running LLMs locally. What most people do is buy the system they can afford and roughly meets their goals and then ball-park VRAM usage by looking at the model size and quantization. For more a detailed analysis, there are several online VRAM calculators. Here's one: https://smcleod.net/vram-estimator/ If you have a huggingface account, you can set your system con…

Just ask Claude to install the most optimum model with a nice chat ui tailored to your wishes. 15'minutes max.

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#297

Post by pawelduda »

finnjohnsen2 wrote: post #261

Since Gemma 4 came this easter the gap from self hosting models to Claude has decreased sigificantly I think. The gap is still huge it just that local models were extremely non-competitive before easter. So now it seems Qwen 3.6 is another bump up from Gemma 4 which is exciting if it is so. I keep an Opus close ofcourse, because these local models still wander off in the wrong direction and fails. Something Opus almo…

How many tokens/s do you get on RTX 5090?

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#298

Post by physicles »

syntaxing wrote: post #132
throwaw12 wrote: post #112

Earlier quoted context omitted.

can you expand more on what you mean by 95%? There are 2 aspects I am interested in: 1. accuracy - is it 95% accuracy of Opus in terms of output quality (4.5 or 4.6)? 2. capability-wise - 95% accuracy when calling your tools and perform agentic work compared to Opus - e.g. trip planning?

1. What do you mean by accuracy? Like the facts and information? If so, I use a Wikipedia/kiwx MCP server. Or do you mean tool call accuracy? 2. 3.6 is noticeably better than 3.5 for agentic uses (I have yet to use the dense model). The downside is that there’s so little personality, you’ll find more entertainment talking to a wall. Anything for creative use like writing or talking, I use Gemma 4. I also use Gemma 4…

I have a Supernote and was looking at different models for handwriting recognition, and I agree that gemma4-26B is the best I’ve tried so far (better than a qwen3-vl-8B and GLM-OCR). Besides turning off thinking, does your setup have any special sauce?

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#299

Post by Havoc »

anonzzzies wrote: post #11

I wish that all announcements of models would show what (consumer) hardware you can run this on today, costs and tok/s.

You can often sorta estimate it but multiply it with like 2/3rds give or take a lot to work out how much vram you need.

27B will fit onto a 24gb card with decent context and a couple GB for operating system to spare at Q4.

tok/s doesn't really have a good way to eyeball it

Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

#300

Post by djfergus »

finnjohnsen2 wrote: post #261

Since Gemma 4 came this easter the gap from self hosting models to Claude has decreased sigificantly I think. The gap is still huge it just that local models were extremely non-competitive before easter. So now it seems Qwen 3.6 is another bump up from Gemma 4 which is exciting if it is so. I keep an Opus close ofcourse, because these local models still wander off in the wrong direction and fails. Something Opus almo…

>> I feel closer to where I should be; writing code should still be free. Both free as in free beer, and free as in freedom.

I’m just pleased by the competition, agree with the ideal of free and local but sustainable competition is key: driving $200 p/m down to a much much lower number.

Post reply on HN