Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

721–730 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#721

Post by Otternonsenz »

jboss10 wrote: post #297
Otternonsenz wrote: post #91

Is there any hope for people that cant even run 27B parameters, Qwen3.6 or otherwise? Are there any quantized models that do well with tool calling at smaller parameter sizes? I do not have a crazy rig, a modest gaming one at that, but in trying to understand more about agents and their capabilities, I am SOL with my 16 GB of RAM and 8GB of VRAM. I can get most small, non tool calling models to perform well, but I've…

I have 8GB VRAM, but 32GB sys ram. I can run qwen 3.6 35B at 30 tok/s. I also use pi, and it's smart enough to extend itself(multishot and maybe a few tries) For you, you could try gemma-4-26B-A4B

Thank you for the recommendation, and so far, it has been working great (within reason, haha). It doesn’t kill my rig when thinking, but it definitely needs more training wheels to nudge it towards the goal.

It seemed to get the idea of my prompt to extend the footer info (I want it to show the model abilities like tool calling or reasoning where the context percent thing is), made a plan and wrote the file, but then got hung up on implementation because it couldn’t figure out how Pi renders that part of the UI in Powershell

So possibly trying a different terminal might help on that front, haha

Re: Qwen 3.6 27B is the sweet spot for local development

#723

Post by etdznots »

cpburns2009 wrote: post #465
lee_ars wrote: post #323

Earlier quoted context omitted.

I'm currently fiddling with a DGX Spark and Qwen3.6-35B-A3B (specifically Qwen3.6-35B-A3B-NVFP4 under vLLM, with EAGLE3 speculative decoding via eagle3-dogacel-vllm), and it's pretty okay in terms of smarts. The speed is relatively usable at about 50 tok/sec with a 256k context window, and it's definitely smart enough to one-shot some basic coding tasks. I had it doing reverse engineering/disassembly of some ancient…

Looping is a common problem with the Qwen models. I've had good luck using --repeat-penalty=1.1 with llama.cpp and 27B. vLLM should have a similar option.

This is the default value!

Re: Qwen 3.6 27B is the sweet spot for local development

#724

Post by icedchai »

Foobar8568 wrote: post #604
ozim wrote: post #589

Earlier quoted context omitted.

DGX Spark everyone is saying performance for the money is not there

I have an access to a DGX spark, and while it performs better than my MacBook Pro (M3 Max), the performance on Qwen and Gemma dense models is dog shit, and not worth it.

Performance with Strix Halo isn't there, either. At least I got mine relatively cheap in 2025, before the run up in prices...

Re: Qwen 3.6 27B is the sweet spot for local development

#725

Post by mswphd »

myzek wrote: post #676
Saris wrote: post #500

Earlier quoted context omitted.

Even with deepseek v4 flash I burned though $5 in credits in a day just playing around with Hermes, and qwen 3.6 35B is significantly more expensive. I can run qwen 3.6 35B on my gaming PC at around 50 tok/s and other than power cost of a tiny bit extra per month, it's hardware I already owned from years ago. I'm not really sure why qwen 3.6 35B is so expensive on openrouter, it seems abnormally high for what hardwar…

How do you run 35B on a gaming PC? I'm trying to go the same route, but I have a 5070Ti with only 16GB VRAM (I bought it for gaming) and I'm not sure how to run anything decent on it. I have 64 GB RAM if that matters

CPU moe offloading. see e.g.

https://www.reddit.com/r/LocalLLaMA/comments/1t9eo83/running...

Re: Qwen 3.6 27B is the sweet spot for local development

#726

Post by aunty_helen »

coder543 wrote: post #477
aunty_helen wrote: post #268

Earlier quoted context omitted.

I was doing some benchmarking last night on 2 3090s. The systems but old but I’m seeing 11tks 27b, 15tks 35b MoE. The limited context is problematic. I’m not exactly sure what it’s got available but hermes was hit and miss on a prospecting job. It does seem to be doing useful work but it’s not API call level quality

> The systems but old but I’m seeing 11tks 27b, 15tks 35b MoE If that's accurate, then you must be doing something wrong/weird. On a single RTX 3090, I'm seeing substantially higher performance. Dual GPU won't necessarily give a ton of performance improvement, but it shouldn't hurt performance. With llama-bench, I just measured Qwen3.6-27B at 41 tok/s and Qwen3.6-35B-A3B at 153 tok/s on one RTX 3090. (Those results a…

Good to know. Might be worth updating the motherboard then, it’s limited in pcie speed.

Re: Qwen 3.6 27B is the sweet spot for local development

#727

Post by icedchai »

toephu2 wrote: post #422
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

I just checked apple's website and configured them: Mac Studio: Ships: 16–18 weeks Mac mini: Ships: 10–12 weeks

Hopefully they're ramping up on the M5 variants.

Re: Qwen 3.6 27B is the sweet spot for local development

#728

Post by icedchai »

pizza234 wrote: post #122
dofm wrote: post #50

Earlier quoted context omitted.

The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all th…

> Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. Cloud models are (much) faster, they don't consume so much power/generate heat, they have much bigger (LLM) context, they're much more precise and they have a much wider (engineering) context of the given problem. Except privacy and use cases that are blocked by cloud models (e.g. reverse engineering), local LLMs are cur…

Same. Local LLMs are fun to experiment with, but when I want generated code of a sufficient quality, I use a cloud LLM.

Re: Qwen 3.6 27B is the sweet spot for local development

#729

Post by androiddrew »

zx76 wrote: post #226

I see a lot of people writing about how expensive the hardware to run these local models is - but see no mentions of the Intel Arc Pro B50/B60/B70 which seem like decent value if you're not interested in Apple kit (as much as anything can be decent value in the current status quo). I just got a B70 with 32GB RAM for the equivalent of $1200 (incl. sales tax and import duties to my non-US location, so presumably it cou…

I try to always mention that AMD ROCm has come a long way. Like the B70 the Radeon AI Pro 9700 has 32GB of DDR6 640GB/s. Also $1300 a card. Very capable cards now in mid 2026. Great for dense models in the 30B range. I'd go strix halo or DGX spark if you want to run the 120B range of MOE models.

Re: Qwen 3.6 27B is the sweet spot for local development

#730

Post by automajicly »

girvo wrote: post #393
SwellJoe wrote: post #342

Earlier quoted context omitted.

I opted to buy a normal 32GB laptop for this very reason. I know how loud and hot the GPUs in my desktop run when running even smallish models like Qwen 27B or Gemma 4 31B (which is a better model for most than Qwen 3.6, despite the benchmarks). I also have a Strix Halo which doesn't get loud, because it has a single huge fan, but it does get hot. So, there's no way a laptop could work as hard as models make them wor…

Gemma is better than Qwen at everything except coding, in all my evaluations. Which is a shame because that is what I use them for!

I have a M1 Macbook Pro...with only 16gb and I struggled with Qwens2.5-14b trying to do large projects. I loved Qwen but I had to try and do something different. So I switched to Gemma4-12b which looking at it now, seems more like a downgrade than an upgrade.Can you refer me to any Qwen coding models that wont choke my poor 16gb and also connect contextually? I need that context. I love the laser point focus, but I need context and basic understanding of that context.
Post reply on HN