Earlier quoted context omitted.
Except they're not available, 3-4 month wait time.
Buy a refurished or 2nd hand one.
Also not really available.
Qwen 3.6 27B is the sweet spot for local development
431–440 of 809 posts
Re: Qwen 3.6 27B is the sweet spot for local development
#432I have been running qwen 3.6 35b a3b with opencode on my macbook pro 16" with m3 max and 64gb ram, and it's been great for local planning and coding. To be honest I have been on and off wishing I had future proofed with the 128gb after seeing how powerful 64gb is. On the other hand, I also haven't run up against a wall with a model that is just slightly larger than qwen.
I've also been running Qwen 3.6 35B A3b on my Windows laptop (64 GB RAM, a 4GB GPU) and it's at least tolerable. It's not fast - a few tokens per second, slower than reading speed - but I can give it a task and come back later. That was a $600 laptop off eBay a few years ago, not a $6,000 machine.
Are these unified memory Macs and giant 24GB desktop GPUs achieving dozens or hundreds of tokens per second commensurate with their 10x-20x cost?
Re: Qwen 3.6 27B is the sweet spot for local development
#433Running 27B dense model on M5 128GB is ok, but one can do better. On M5 128GB one can make use of the ram and use sparse MoE. For example, DeepSeek-V4-Flash will fit, served by DwarfStar ( https://github.com/antirez/ds4 ). One will probably improve 2x the token/sec speed, given DS4F 13B activated params in the MoE are ~1/2 of the ~27B of the dense Qwen. 27B Of the Qwen fit even on a cheaper 24GB card, e.g. amd 7900xt…
This is discussed in the article:
"My personal impression is that within these quantizations Qwen 3.6 27B is as good as (or maybe slightly better than) DwarfStar4. Though, I won’t be surprised if for longer context projects DS4 has an edge."
Re: Qwen 3.6 27B is the sweet spot for local development
#434I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…
They really need to release those updated Studios already.
Since they've reduced the max RAM on current Studios from 512GB to 96GB, I'm not holding my breath.
Re: Qwen 3.6 27B is the sweet spot for local development
#435Earlier quoted context omitted.
That’s 24GB VRAM. Not enough to run a 27B model at a useful quant+context size.
You can run 8bit 27B models at 24GB, it's definitely enough for the model size.
I'm running qwen 3.6 27b at 8bit quantization and 262k context. It takes 53gb of vram on my system.
Re: Qwen 3.6 27B is the sweet spot for local development
#436Earlier quoted context omitted.
DGX Spark is one, but really depends on how much you want to spend
273GB/s bandwidth vs 614 GB/s of the M5 Max. And you're getting a whole laptop. $5k for DGX Spark as well.
Prompt processing time is better on the spark, which aligns more with coding (more reading than writing).
I spent less than $4k, OEM are better boxes for cooling, no apple markup, I get a real Linux system for stuff like k3s.
Re: Qwen 3.6 27B is the sweet spot for local development
#437The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...
The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all th…
> I totally struggled to find the right frame of mind to explore any of this stuff without feeling defeated and bamboozled. Because it's just huge, exhausting, jargon-drenched, unknowable, and I am over the hill at fifty-plus.
Hello, my brother, just know that you have a fellow passenger in life at the same age who thinks the same thing. I agree that the local stuff is helping my understanding a LOT.
However, my gut feel as someone who got to experience the TeleBomb after the DotBomb is that the obfuscation is INTENTIONAL--it's neither you nor your age. I remember asking people to explain to me what the OC-768 startup endgame was when roughly 10 OC-768 links could carry the world's traffic at the time--and everybody giving me blank looks. The AI Bubble has the EXACT same feel as the Telecom Bubble--just bigger.
What I really wish is that I could find a VPS-type provider where I could toss things into their NVIDIA/AMD machines for an hour or two. Alas, all of the providers seem to want massive paperwork and huge minimum purchases.
I can't wait for the bubble to pop so that we mere mortals can finally build with this stuff.
Re: Qwen 3.6 27B is the sweet spot for local development
#438Earlier quoted context omitted.
If you want to do coding with a local LLM your best bet is a 6 year old Nvidia 3090 which is substantially more powerful than the highest end overhyped Apple product for 1/5th the price.
An M1 Ultra has 800gbps unified memory. It’s nothing to do with Apple, it’s their microarchitecture. They’re just about the only game in town with high-bandwidth memory if you want >24GB (for less than $10k, anyway).
A 5090 gets you 32GB with 1.8 TB/s of memory bandwidth for ~$4k, RTX A6000 gets you 48GB at 768 GB/s for ~$3.5k, 2x 3090 gets you 48GB for $2000 or so, and if you're willing to go into the wilderness, there are much cheaper options like the AMD MI50.
Re: Qwen 3.6 27B is the sweet spot for local development
#439I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…
Try using DwarfStar 4 and use the --power flag:
https://github.com/antirez/ds4#reducing-heat-power-usage-and...
Re: Qwen 3.6 27B is the sweet spot for local development
#440Earlier quoted context omitted.
When's the last time you shoed a horse? The reason I delegate so much of local LLM installation and administration to Claude Code is simply because there's no point learning practical things that will work completely differently in a couple of years, or in memorizing procedures that I'll forget long before I need to perform them again. No longer having to sweat all the details is a Good Thing, not a Bad Thing.
> When's the last time you shoed a horse? That skill died too, so what's your point?
Skills sometimes do that. What's your point?