Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

591–600 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#591

Post by pletnes »

mft_ wrote: post #366
JeremyNT wrote: post #121

Earlier quoted context omitted.

I also don't understand why people in this price bracket are buying Mac laptops instead of desktop computers with GPUs? Just to flex that it's portable?

(I'm not one of the people you're speaking of with a 128gb M5 but) if you want to run one of the medium-sized open-weights models (Qwen 27b, 35b, Gemma 4 26b, 31b) or larger, you get into an interesting optimisation space. * yes, you can run it on an older/smaller GPU plus system RAM but performance will suffer * if you want optimal GPU performance you need the model in VRAM plus context, so 24GB (3090, 4090) or 32GB…

And with a mac, there are no cuda drivers to fiddle with.

Re: Qwen 3.6 27B is the sweet spot for local development

#592

Post by ant6n »

bensyverson wrote: post #26

The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...

Doesnt it run on the Macbook Neo... just slower?

Re: Qwen 3.6 27B is the sweet spot for local development

#593

Post by mips_avatar »

tasoeur wrote: post #580
mips_avatar wrote: post #358

I think the sweet spot right now is 2x 3090s and a pcie 4 motherboard with 64-128 gb of ddr4 ram, you can build this right now for $3k and it runs qwen 27b/35b stupid fast at int4.

I know how to build PCs but suck at picking parts, would you happen to have a recommended build or links to people who've done similar ones? Heck I'll click on an affiliate link to support the author of the build :-)

Here's my build! https://jonready.com/blog/posts/local-llm-rig.html

I love it because the watercooled 3090s are completely silent even under load. Facebook marketplace is definitely the move for a lot of the parts unfortunately, since you ideally would have higher end parts that are 2-3 years old.

Re: Qwen 3.6 27B is the sweet spot for local development

#594

Post by sixothree »

nirvdrum wrote: post #588
sixothree wrote: post #586

Earlier quoted context omitted.

Wait. Did they raise their prices a second time?

Probably USD vs CAD. The parent posted a /ca/ link, which will look really similar to /us/, but the prices will all appear to be higher.

Ah. Thank you. It seemed pretty sticky too, navigating the items via my previous orders even persisted the currency.

Re: Qwen 3.6 27B is the sweet spot for local development

#595

Post by 2Gkashmiri »

aurareturn wrote: post #376
codazoda wrote: post #367

Earlier quoted context omitted.

Today the Mini tops out at 48GB. Gotta go to the Studio to get 64GB.

Don't buy the Mini or Studio. Both have the M4 which lacks the Neural Accelerators, making prompt processing ~3-4x slower.

Apple Mac Studio (M3 Ultra Chip/28 CPU, 60 GPU/96 GB/1 TB

How is this config?

Re: Qwen 3.6 27B is the sweet spot for local development

#597

Post by danielrmay »

jimmaswell wrote: post #546

My partner has been trying various models on our server but we haven't gotten anything to run at a usable speed. Q30H engineering sample (Xeon 8570) with two cpus, 56 cores per CPU, 768GB DDR5 RAM running at 5600MHz, two old 3090s in it at the moment with an NVLink and we could put our third in there. We built this server before the prices skyrocketed because we happened across some Tyan boards on Woot that were absu…

Yes, this should be a monster machine. Ampere is an older generation, so I expect that's where some of your issues have been

Re: Qwen 3.6 27B is the sweet spot for local development

#600

Post by bpye »

dofm wrote: post #50
bensyverson wrote: post #26

The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...

The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all th…

> The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it.

Seems like a GPU with 12GB+ VRAM is going to be a much more affordable way to achieve that? Even a B580 should get reasonable perf there.

Post reply on HN