Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

681–690 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#681

Post by m3kw9 »

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

Your MacBook will not last running current big LLMs on these hardware. The heat will wear on it.

Re: Qwen 3.6 27B is the sweet spot for local development

#682

Post by m3kw9 »

dofm wrote: post #50
bensyverson wrote: post #26

The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...

The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all th…

What’s the use of a 4gb Gemma other than to just play with it ?

Re: Qwen 3.6 27B is the sweet spot for local development

#683

Post by zkmon »

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

The Q6_K gguf fits nicely on a 24GB GPU. That's amazing.

Re: Qwen 3.6 27B is the sweet spot for local development

#684

Post by brandensilva »

PeterStuer wrote: post #617
brandensilva wrote: post #547

Earlier quoted context omitted.

Thoughts on a M5 Ultra 768GB if it drops? What's the price to make it worth it for you over a spark cluster? I'm wanting to run Kimi 2.6/2.7 GGUF on it and just slap it in the server rack, but trying to decide if a spark cluster makes more sense.

The M3 with 512GB is currently sitting at around 30K, used. You can extrapolate from there.

[dead]

Re: Qwen 3.6 27B is the sweet spot for local development

#685

Post by urbsgpw »

tedivm wrote: post #402
c7b wrote: post #379

Earlier quoted context omitted.

This. Do consider local LLMs, but set aside a dedicated machine for it. Connect via VPN or reverse proxy. If it's not a Mac them I'd also put a server distro on it. No need for a desktop environment, save your RAM.

I have a Linux box with two 3090s and it's been great for running Qwen3.6 27b. I lowered the power on each card down to 250w, and then built a small ducting/fan system to vent the waste heat outside. The machine is pretty much silent, and I'm still getting 110 tokens per second out of it for coding tasks. https://github.com/tedivm/qwen36-27b-docker

But is Qwen3.6 27B actually worth this investment? If I had to guess you still use SOTA for architectural/planning work?

Re: Qwen 3.6 27B is the sweet spot for local development

#686

Post by chiply314 »

andai wrote: post #333
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

> The reason is simple: your fingers will burn and your head will explode from the noise. So, just buy a mac mini and put it in the other room? ( Like everyone was doing in February? :) I've been running coding agents on my laptop in yolo mode for the past half year or so (though mostly not local ones, laptop too slow!) and the way I'm doing that without terror is that I just gave them their own Linux user "agent". T…

I think more bought them to run their Clawed on it but still with external LLM calls.

There should be a lot more content on setups and best practices etc. if these macs would be used with local models only.

Re: Qwen 3.6 27B is the sweet spot for local development

#689

Post by ntcho »

v1ne wrote: post #623
alemanek wrote: post #564

Earlier quoted context omitted.

If your workflow benefits from the speed it quickly pays for itself when factoring in developer salaries here in the US. I recently switched companies and they bought me an M5 Max 128GB as my dev machine. Builds and local test runs are 3 times faster than the Windows laptop option. The machine will pay for itself just based on that within 3 months. I can spin up a local kubernetes cluster and do full integration test…

Don't worry. Once IT Security discovers that they miss their trusty endpoint security products on your Mac, they'll add it and you'll be in the same ballpark as the Windows machine. Been there, received that, and learnt that Microsoft Defender exists for macOS, too.

Gosh, the last sentence is the most terrifying thing I’ve read today.

Re: Qwen 3.6 27B is the sweet spot for local development

#690

Post by ThunderSizzle »

this_was_posted wrote: post #679
ThunderSizzle wrote: post #659

Earlier quoted context omitted.

Agreed. I have a single 9700 and I'm able to fit Q6 27B at 30tps or Q5 35B at 100tps very easily via llamacpp running vulkan. The results are impressive considering the amount of people trashing AMD and still trying to recommend 3090s. I hope to buy a 2nd one at some point, but I also hate the version hell of vLLM, the R9700, the ROCM version, and Qwen3.6 all not agreeing with each other. I haven't gotten vLLM to run…

I'm getting around 45 tps on a single r9700 for Q6 27B with build b9811 ( using https://github.com/kyuz0/amd-r9700-ai-toolboxes ) with the following parameters: llama-server -hf unsloth/Qwen3.6-27B-MTP-GGUF:Q6_K -c 135000 -ngl 999 -np 2 -t 16 --temp 0.0 --top-p 0.95 --top-k 20 --min-p 0.00 -b 4096 -ub 4096 --chat-template-kwargs '{"preserve_thinking": true}' -fa 1 --spec-type draft-mtp --spec-draft-n-max 2

I'll give 27B-MTP a try. I think I can tolerate 45 tps if the results are technically better. 35B is pretty good, but definitely shows it's inabilities at times (probably either due to the heavy caching quantization I'm doing, or the heavy model quantization vs what 2 GPUs could run).

My biggest gripe is that both pi and opencode seem to have trouble parsing the thinking blocks at times, and the model sometimes cuts-off mid-thinking or prints out weird character tokens at times. I don't know if that's because of llamacpp, pi/opencode, or qwen3.6, or some weird combination of them all, as I haven't investigated that problem fully yet.

Post reply on HN