Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

551–560 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#551

Post by trollbridge »

geophile wrote: post #320
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

That's exactly what I'm doing -- Mini M4 Pro 64GB, qwen3.6. My hearing is not great, but I think I would have noticed the fan, and I have never heard it. In fact, I had to google to find out if it even has a fan.

I'm still kicking myself for buying a 32GB M1 Max Studio two years ago when it wouldn't have been that difficult to get a 64GB instead.

Re: Qwen 3.6 27B is the sweet spot for local development

#552

Post by christina97 »

jimmaswell wrote: post #546

My partner has been trying various models on our server but we haven't gotten anything to run at a usable speed. Q30H engineering sample (Xeon 8570) with two cpus, 56 cores per CPU, 768GB DDR5 RAM running at 5600MHz, two old 3090s in it at the moment with an NVLink and we could put our third in there. We built this server before the prices skyrocketed because we happened across some Tyan boards on Woot that were absu…

Start with a quant, you can run the Qwen 27B model at 4-bit on one 3090, presumably 6/8-bit on 2x3090.

Re: Qwen 3.6 27B is the sweet spot for local development

#553

Post by stevenhubertron »

satvikpendem wrote: post #529
ekianjo wrote: post #505

Earlier quoted context omitted.

gemma is also worse for tool calling. not just coding

That is because they use a different tool calling format than most other models. Unsloth quants fix this in their Gemma releases.

Can you say a bit more about this? The bad tool calling has made me give up on using Gemma for my Hermes and a personal recipe site. I have only downloaded from Ollama.

Re: Qwen 3.6 27B is the sweet spot for local development

#554

Post by kevinrineer »

whynotmaybe wrote: post #59
spike021 wrote: post #35

Earlier quoted context omitted.

Certainly won't work on my M4 Pro with 24GB lol

I feel you! Sent from my 8gb M2 Mac mini.

I'm still rocking my nvidia 2060, which I had purchased for $400 at the time.

I struggle to imagine purchasing multiple 1k+ cards on my own dime.

Re: Qwen 3.6 27B is the sweet spot for local development

#555

Post by trey-jones »

mullen wrote: post #245
oceanplexian wrote: post #219

Earlier quoted context omitted.

Honestly your best bet is to buy a $20 Claude subscription, ask Claude to set it all up with Pi and llama.cpp and come back in 20 minutes after a cup of coffee. This is also a good idea because it will help set expectations of what a local model can do vs. a frontier model.

This is what I did after struggling to get llama.cpp working at a decent speed on my M1 Macbook. The secret is to very specific with your needs and targeted in what you are using llama.cpp for. Mine setup is just about strictly for qwen3-coder and now, I get a fairly decent speed out of it. I also installed Cursor to check Claude and it all worked out well.

I don't know why we're even talking about Qwen3.6 for writing code when qwen3-coder exists. My experience is there's no contest. I'm using 30b with 96k context on a dedicated server.

Re: Qwen 3.6 27B is the sweet spot for local development

#556

Post by julianlam »

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

Very surprised an Apple device can have some atrocious ventilation design.

I'm running this model on a Framework 13 and the chassis barely heats up at all while running full tilt.

Re: Qwen 3.6 27B is the sweet spot for local development

#557

Post by jtbaker »

murderfs wrote: post #438
jnovek wrote: post #258

Earlier quoted context omitted.

An M1 Ultra has 800gbps unified memory. It’s nothing to do with Apple, it’s their microarchitecture. They’re just about the only game in town with high-bandwidth memory if you want >24GB (for less than $10k, anyway).

A 5090 gets you 32GB with 1.8 TB/s of memory bandwidth for ~$4k, RTX A6000 gets you 48GB at 768 GB/s for ~$3.5k, 2x 3090 gets you 48GB for $2000 or so, and if you're willing to go into the wilderness, there are much cheaper options like the AMD MI50.

The RTX 5000 Pro 72GB seems like kind of a sleeper to me, and sips https://www.microcenter.com/product/709071/pny-nvidia-rtx-pr...

Re: Qwen 3.6 27B is the sweet spot for local development

#558

Post by fossheart »

> I recommend llama.cpp - a direct, open source tool that allows running models on various devices. You don’t need Ollama, and frankly - I would recommend against using that on ethical grounds.

> https://sleepingrobots.com/dreams/stop-using-ollama/

I had faced roadblocks while integrating with openclaw using ollama (Was trying to experiment with `qwen3-vl:2b`). I was tracking the issue back to openclaw at that time, I didn't even consider investigating ollama.

I attached a threads post here where I'm talking to meta ai to expand on both scenarios (not to use ollama, but llama.cpp & my take on the why this is the way it is - ie. commercial gains)

https://www.threads.com/@riojos/post/DaMXIs4k4w8

Re: Qwen 3.6 27B is the sweet spot for local development

#559

Post by dd8601fn »

827a wrote: post #518
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

Apple does not sell a 64GB variant of the M4 Mac Mini. IIRC they never have; its always capped out at 48GB. If you were planning on getting an M5 128GB; just get a DGX Spark (~$4500) or a 5090-equipped machine (~$4500) plus a Macbook Air (~$1500). You'll come in below the M5 Max 128 pricing (~$6700+ USD) and be happier for it.

I'm using a 64GB M4 Mac Mini.

They pulled them a month or two ago, right after I bought it.

Re: Qwen 3.6 27B is the sweet spot for local development

#560

Post by fouc »

trey-jones wrote: post #555
mullen wrote: post #245

Earlier quoted context omitted.

This is what I did after struggling to get llama.cpp working at a decent speed on my M1 Macbook. The secret is to very specific with your needs and targeted in what you are using llama.cpp for. Mine setup is just about strictly for qwen3-coder and now, I get a fairly decent speed out of it. I also installed Cursor to check Claude and it all worked out well.

I don't know why we're even talking about Qwen3.6 for writing code when qwen3-coder exists. My experience is there's no contest. I'm using 30b with 96k context on a dedicated server.

For agentic workflows like tool use, editing codebases, multi-turn debugging?
Post reply on HN