Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

161–170 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#161

Post by stymaar »

boutell wrote: post #123
stymaar wrote: post #79

Earlier quoted context omitted.

> The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Qwen3.6-27B would be faster on a 3090 that costs around $1000-1200 though so I don't think it's a good counter-argument. Op just happened to have that MacBook, but it doesn't mean it's necessary to run the model.

That 3090 is going to burn 750W and it will still cap you at a 4 bit quant and ~48K context. Here's someone who worked through it: https://github.com/noonghunna/qwen36-27b-single-3090 Flies though (50-70tps is impressive for a model this smart) I went through roughly the same process to get it working on my M2 Macbook Pro... at awful speeds of course, since models like this one are mostly bound by memory bandwidth.

> That 3090 is going to burn 750W

The 3090's TPD is 350W, but given that LLM's token generation isn't compute bound, people usually undervolt these cards to reduce power consumption. IIRC you can get as low as 200-250W without any degradation. Caveat these figures are without speculative decoding and at batch size =1.

Re: Qwen 3.6 27B is the sweet spot for local development

#162

Post by colinsane »

verdverm wrote: post #72

Qwen's new AgentWorld model is good too: https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B I'm running the NVFP4 alongside Gemma4 at the same quant on an OEM Spark

AgentWorld is _fantastic_. i just migrated "down" from the 122B A10B qwen model to agentworld (35B A3B) because it feels as capable, easier to steer, and it's 3x faster.

also i like that if i drop more sophisticated tools into my harness (e.g. any of the NLP/RAG-based search tools in place of grep/rg), the agent will actually reach for them and make progress faster; previous models have been reluctant to embrace new tools.

Re: Qwen 3.6 27B is the sweet spot for local development

#163

Post by blopker »

iwontberude wrote: post #90
blopker wrote: post #80

I've been working with local models for the past year. There's so many possibilities, but I don't think coding is one. Coding requires so many layers beyond inference; I spent so much time trying to replicate what Claude Code does end to end locally. Understanding all the layers and keeping up with the advancements feels like a slog. Even this article messes up and misunderstands what some of the settings are doing.…

> I don't think coding is one Certainly this is falsifiable easily by any of us doing it on a regular basis > Qwen stuck in thought loops This does happen when context is not managed effectively; creating plans, using subagents and compactions strategically resolves this

Sure, local coding is clearly _possible_, but it's not practical for most people. I've yet to see a reliable setup, if you have one, I'd love to see.

> creating plans, using subagents and compactions

Yes, these are all things that Claude Code does for you. However, for the thought loop issue, these are not the fixes. The canonical fix is to limit the number of thought tokens (llama.cpp's `--reasoning-budget`) or try to mess with the various penalty parameters. In any case, it's not a solved problem as far as I can tell.

Re: Qwen 3.6 27B is the sweet spot for local development

#164

Post by wilsonnb3 »

throw1234567891 wrote: post #125
bensyverson wrote: post #26

The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...

But the tokens or credits are gone. MacBook stays. You can run other models on the same MacBook. What I read people burn every month on saas… for that money you break even on that MacBook in 5 months. Edit: it’s not just “data privacy”, when you are using Claude, you are shipping EVERYTHING to Anthropic. It’s crazy.

Companies are already shipping everything to Microsoft or Google and 17 other companies, just the cost of doing business.

Re: Qwen 3.6 27B is the sweet spot for local development

#165

Post by blueside »

i have been trying several open source models for the last few years. running qwen 3.6 27b on my 4090 is the first local llm i have used that made me start to second question if anthropic and openai are actually worth the (already) insane valuations.

don't get me wrong, the frontier models are leaps and bounds ahead of what qwen/kimikgemma are doing - but i don't need to drive a ferrari to the grocery store everytime either.

Re: Qwen 3.6 27B is the sweet spot for local development

#167

Post by charcircuit »

coldtea wrote: post #147
ddalex wrote: post #81

Earlier quoted context omitted.

I just got Claude to download and install all the models and servers and agents and prepare all the launch scripts for me... no need to learn, just ask it to do it for you

> no need to learn, just ask it to do it for you And that's how skills die.

Except with AI models it's possible to make a backup of them creating a permanent artifact of a skill.

Re: Qwen 3.6 27B is the sweet spot for local development

#168

Post by wahnfrieden »

throw1234567891 wrote: post #125
bensyverson wrote: post #26

The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...

But the tokens or credits are gone. MacBook stays. You can run other models on the same MacBook. What I read people burn every month on saas… for that money you break even on that MacBook in 5 months. Edit: it’s not just “data privacy”, when you are using Claude, you are shipping EVERYTHING to Anthropic. It’s crazy.

It's much slower, and often quantized

Re: Qwen 3.6 27B is the sweet spot for local development

#169

Post by dofm »

wahnfrieden wrote: post #160
dofm wrote: post #141

Earlier quoted context omitted.

Again, I would not argue against any of this. And I can't say that I won't switch to openrouter (even just for the same models) at some point. But one of the things I have found about my own process learning is that some lessons only come to you when you make yourself available to them. And if that means doing things the difficult way, that is what you should do.

Difficult... and wastefully expensive

I mean, it's a (secondhand) computer I bought for other tasks (processing very large photos, compiling large apps quickly). It's running all the time. It can also run LLMs when I want to.

The rest of my life is ultra-frugal so I am relaxed about this.

Re: Qwen 3.6 27B is the sweet spot for local development

#170

Post by psychoslave »

pizza234 wrote: post #122
dofm wrote: post #50

Earlier quoted context omitted.

The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all th…

> Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. Cloud models are (much) faster, they don't consume so much power/generate heat, they have much bigger (LLM) context, they're much more precise and they have a much wider (engineering) context of the given problem. Except privacy and use cases that are blocked by cloud models (e.g. reverse engineering), local LLMs are cur…

Anything done local will likely come at higher cost and at scale with less energy efficiency and commodity, with less possibility to fine tune engineer deeply on wider horizon of issues.

That's never the point of keeping local alternatives though.

Post reply on HN