Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

781–790 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#781

Post by c16 »

c16 wrote: post #638
jasonjmcghee wrote: post #579

Earlier quoted context omitted.

I'm surprised no one has else has mentioned - low power mode. With no speculative decoding, using high power mode, I get 80 t/s on 35B A3B - and it gets hot and spins up. On low power mode I get 38 t/s - no fans, cool to warm laptop. If you currently don't use speculative decoding and you start using it, it can nearly offset the difference between high and low power, and it's night and day experience. I almost always…

Will give this a try later. Enjoy working with A3B Coder, but the heat coming out my 32gb M5 is a lot. This might be the trick - Thanks!

Update: This works super well. Thanks.

Re: Qwen 3.6 27B is the sweet spot for local development

#782

Post by jnovek »

angoragoats wrote: post #526
jnovek wrote: post #258

Earlier quoted context omitted.

An M1 Ultra has 800gbps unified memory. It’s nothing to do with Apple, it’s their microarchitecture. They’re just about the only game in town with high-bandwidth memory if you want >24GB (for less than $10k, anyway).

Yeah this is just not the case at all; a 5090 or any of the recent nvidia workstation cards all fit this criteria. Also, while memory bandwidth is important, it isn’t the only consideration. Apple’s architecture has memory bandwidth equal to a mid-range consumer GPU, but its GPU speed is much, much worse than, say, a 5080 or 5090. This translates into e.g. much slower time to first token on Mac systems compared to de…

You are correct, for me TTFT is about 50% longer on an M1 Ultra than on a GPU, but you can also get an M1 Ultra Studio with 32GB for I’m wrong about the prices, but $4k-$5k for a GPU is way out of hobbyist range where a computer that I can use for other stuff is accessible.

Re: Qwen 3.6 27B is the sweet spot for local development

#783

Post by angoragoats »

jnovek wrote: post #782
angoragoats wrote: post #526

Earlier quoted context omitted.

Yeah this is just not the case at all; a 5090 or any of the recent nvidia workstation cards all fit this criteria. Also, while memory bandwidth is important, it isn’t the only consideration. Apple’s architecture has memory bandwidth equal to a mid-range consumer GPU, but its GPU speed is much, much worse than, say, a 5080 or 5090. This translates into e.g. much slower time to first token on Mac systems compared to de…

You are correct, for me TTFT is about 50% longer on an M1 Ultra than on a GPU, but you can also get an M1 Ultra Studio with 32GB for I’m wrong about the prices, but $4k-$5k for a GPU is way out of hobbyist range where a computer that I can use for other stuff is accessible.

$3k for a 5090 fits your criteria and is in the same category of "you can use it for other stuff" (gaming), and is going to run circles around the M1 Ultra Mac Studio.

Or you can spend ~$2k on a pair of 3090s, which gives you 48GB of memory and will also be significantly faster than the M1 Ultra.

IMO there is no situation in 2026 where buying a 32GB M1 Ultra is the right move.

edit: for the folks brave enough to try them there are also additional aftermarket/modded options; I've seen both 4080 Super cards upgraded to 32GB of memory, and 4090 cards upgraded to 48GB. The former comes in at $2k-2.5k and would still be faster than the M1 Ultra.

Re: Qwen 3.6 27B is the sweet spot for local development

#784

Post by QuercusMax »

hypfer wrote: post #647
dofm wrote: post #404

Earlier quoted context omitted.

Yes — there is a setting for that isn't there. And as soon as you realise there's a setting for that, you have new knowledge. Qwen barely needs any of Opencode's prompt, in my experience; I think I cut it down to about three general lines I found by googling. Mainly you need only a pre-amble to make sure that the plan mode, plan switch and build mode prompt fragments make sense. Gemma 4 also needs almost nothing at a…

What are those 3 lines you've cut it down to?

I've been using pi-coder the couple days with Qwen, and its minimal prompt works MUCH better than OpenCode's bloated prompt.

Re: Qwen 3.6 27B is the sweet spot for local development

#785

Post by automajicly »

SwellJoe wrote: post #760
automajicly wrote: post #730

Earlier quoted context omitted.

I have a M1 Macbook Pro...with only 16gb and I struggled with Qwens2.5-14b trying to do large projects. I loved Qwen but I had to try and do something different. So I switched to Gemma4-12b which looking at it now, seems more like a downgrade than an upgrade.Can you refer me to any Qwen coding models that wont choke my poor 16gb and also connect contextually? I need that context. I love the laser point focus, but I n…

I don't think "large projects" is realistic with a model that fits in ~8GB (I'm assuming you run stuff other than the model). And, Gemma 4 12B QAT at 4-bits is surely the smartest model in its size, but it shines at vision tasks rather than agentic tasks (though it is a good tool user and can do stuff like research, it's obviously not aimed at code). You can almost always find free models on OpenRouter. Google AI Stu…

WOW. That was alot of models. I will be sure to look into those. What caught my eye was the North Mini Code. How is that at reason and context? I guess I will just go see. Thank you for the insight. I am building and learning at the same time, and I really need a model that wont chew up my poor little M1, but at the same time will understand my direction, my context, and also my explanation of need and then be ...' oh, I know what you mean..' then help vibe with me to create it. Qwen was a code model but hallucinated and was never really good at contextual understanding. Gemma seems to get it but isnt necessarily a coding model.(As you said) And I just cant be bogged down with all these fees to get what I need on my journey. So onward on the model search I suppose. Thank you very much for the time.

Re: Qwen 3.6 27B is the sweet spot for local development

#786

Post by automajicly »

SwellJoe wrote: post #760
automajicly wrote: post #730

Earlier quoted context omitted.

I have a M1 Macbook Pro...with only 16gb and I struggled with Qwens2.5-14b trying to do large projects. I loved Qwen but I had to try and do something different. So I switched to Gemma4-12b which looking at it now, seems more like a downgrade than an upgrade.Can you refer me to any Qwen coding models that wont choke my poor 16gb and also connect contextually? I need that context. I love the laser point focus, but I n…

I don't think "large projects" is realistic with a model that fits in ~8GB (I'm assuming you run stuff other than the model). And, Gemma 4 12B QAT at 4-bits is surely the smartest model in its size, but it shines at vision tasks rather than agentic tasks (though it is a good tool user and can do stuff like research, it's obviously not aimed at code). You can almost always find free models on OpenRouter. Google AI Stu…

Sorry, I also forgot to say, to the rest of your comment: Its actually 16gb M1. And yes the entire operation is, all running at the same time. Kali/UTM with an mcp server pointed at my LMstudio with gemma loaded as the local server. And maybe some spotify to lighten the mood while I work. But I learned not to have multiple things running, REAL QUICK. So ultimately I am running pretty healthy and no sort of crashes or choke points. I ran a 14b QWEN in LM Studio w/ no issue. Any higher and I think I would fall off a cliff somewhere.

Re: Qwen 3.6 27B is the sweet spot for local development

#787

Post by dang »

h4ny wrote: post #774
dang wrote: post #373

Earlier quoted context omitted.

Yikes, you broke the site guidelines badly with this post. Could you please review https://news.ycombinator.com/newsguidelines.html and stick to them? You're welcome to make your substantive points thoughtfully, just not aggressively.

Noted, reviewed, and understood. I'll be more mindful. Sorry for getting caught in the moment.

Appreciated!

Re: Qwen 3.6 27B is the sweet spot for local development

#789

Post by georgeven »

sixdimensional wrote: post #587
Dig1t wrote: post #112

Earlier quoted context omitted.

How did you buy 3 V100's for $1500??

Not OP and just guessing, but probably SXM2 GPU modules for the V100. Those can be acquired fairly inexpensively, but there is work to do to get them working together and the V100 has some limitations on the types of models you can run.

PCIE 16 gig is <300 dollars these days a piece

Re: Qwen 3.6 27B is the sweet spot for local development

#790

Post by dofm »

QuercusMax wrote: post #784
hypfer wrote: post #647

Earlier quoted context omitted.

What are those 3 lines you've cut it down to?

I've been using pi-coder the couple days with Qwen, and its minimal prompt works MUCH better than OpenCode's bloated prompt.

I was going to reply with my answer but I tested it again and realised that actually in some cases the custom prompt is somehow being ignored, so I don’t truly know how good it actually is in opencode.

Poking around in the convoluted opencode source or the overstuffed github issues to try to make sense of exactly what is happening has convinced me to switch to pi as well, simply to have a thing I can more fully understand. I am using paseo anyway for the GUI on the Mac, so switching agents is easy enough and pi has always seemed like it might be the right choice in principle. Initially I just picked opencode as the easiest path but now it feels like the time.

Post reply on HN