Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

521–530 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#521

Post by lowbloodsugar »

iagooar wrote: post #410
ActorNightly wrote: post #346

Earlier quoted context omitted.

>If you want to run Qwen3.6 27B / 35B at its best, get a MacMini M4 with 64GB of RAM and put it in the basement Im sorry, but its time to start calling Apple sycophants out. Stop trying to push your tech jewelry on other people. You only buy those computers because they are Apple, you don't know anything about computing or running LLMs, you don't do any real work, so you should probably not give advice on what to buy…

I am not going to flag you, I am much OK with having good arguments. I just purchased a Mac Mini M4 Pro 64GB for $3k - 2nd hand of course. I am not a hater of Nvidia and I am planning on building a workstation based on RTX cards. You clearly do not seem to understand how convenient the MacMini actually IS - the form factor, how quiet it is, how durable it is, how well it integrates with other Macs, how well it works…

If you are in Apple ecosystem, and have reasons to own one besides inference, then buying a used Mac mini pro isn’t such a bad idea. I just bought a regular Mac mini just to provide a nice front end to my Ubuntu workstation. But if all you want is inference, then a cheap PC with a 32gb 9700 (or two!) in it is far cheaper. This specific thread was about someone who already has a MacBook. A cheap PC and GPU pairs well. Or a spark: slower but more memory. Or fuck it! Get a 5090 or a 6000!

Re: Qwen 3.6 27B is the sweet spot for local development

#522

Post by angoragoats »

xd1936 wrote: post #251
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

Apple does not currently sell a Mac Mini with 64GB RAM.

They did until 4 days ago, so I’d forgive the OP for not knowing that the option was discontinued.

Re: Qwen 3.6 27B is the sweet spot for local development

#523

Post by angoragoats »

chorizo wrote: post #248
oceanplexian wrote: post #207

Earlier quoted context omitted.

If you want to do coding with a local LLM your best bet is a 6 year old Nvidia 3090 which is substantially more powerful than the highest end overhyped Apple product for 1/5th the price.

That’s 24GB VRAM. Not enough to run a 27B model at a useful quant+context size.

So buy two.

Re: Qwen 3.6 27B is the sweet spot for local development

#526

Post by angoragoats »

jnovek wrote: post #258
oceanplexian wrote: post #207

Earlier quoted context omitted.

If you want to do coding with a local LLM your best bet is a 6 year old Nvidia 3090 which is substantially more powerful than the highest end overhyped Apple product for 1/5th the price.

An M1 Ultra has 800gbps unified memory. It’s nothing to do with Apple, it’s their microarchitecture. They’re just about the only game in town with high-bandwidth memory if you want >24GB (for less than $10k, anyway).

Yeah this is just not the case at all; a 5090 or any of the recent nvidia workstation cards all fit this criteria.

Also, while memory bandwidth is important, it isn’t the only consideration. Apple’s architecture has memory bandwidth equal to a mid-range consumer GPU, but its GPU speed is much, much worse than, say, a 5080 or 5090. This translates into e.g. much slower time to first token on Mac systems compared to dedicated GPUs.

Re: Qwen 3.6 27B is the sweet spot for local development

#527

Post by shockembopper »

bensyverson wrote: post #26

The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...

I’ve got qwen3.6 27b running on my media server atm. Given that I built on top of what I already had, it didn’t cost me nearly that amount. I’ve been running 2x 5060 ti 16gbs, and when using text only and nvfp4, I can run the model with 200k context length and roughly 50-60 toks. It’s very good, and costed me about $800 after buying the gpus from microcenter.

Re: Qwen 3.6 27B is the sweet spot for local development

#528

Post by FuckButtons »

JeremyNT wrote: post #121
doodlesdev wrote: post #83

I feel like I'm going insane seeing people buy these 128gb MBP for thousands of dollars to run models that are objectively much worse than SOTA and spending so much more. The amount spent on a 128gb M5 MAX can buy you a damned new car here. What the hell am I missing? Are developers in other countries living in such different worlds? (I'm aware the price is, in absolute terms, more expensive where I live compared to…

I also don't understand why people in this price bracket are buying Mac laptops instead of desktop computers with GPUs? Just to flex that it's portable?

The fact that I can take it with me? That I don’t need internet to still have access to deepseek? The fact that electricity is expensive and an mbp uses ~10% of the power that an equivalent vram set up would using gpu’s. Also, in order to get the same vram I would need to spend a similar amount, but wouldn’t also have a machine that was useful for other workloads that need a huge amount of ram.

Re: Qwen 3.6 27B is the sweet spot for local development

#529

Post by satvikpendem »

ekianjo wrote: post #505
girvo wrote: post #393

Earlier quoted context omitted.

Gemma is better than Qwen at everything except coding, in all my evaluations. Which is a shame because that is what I use them for!

gemma is also worse for tool calling. not just coding

That is because they use a different tool calling format than most other models. Unsloth quants fix this in their Gemma releases.
Post reply on HN