Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

281–290 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#281

Post by sanderjd »

pizza234 wrote: post #122
dofm wrote: post #50

Earlier quoted context omitted.

The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all th…

> Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. Cloud models are (much) faster, they don't consume so much power/generate heat, they have much bigger (LLM) context, they're much more precise and they have a much wider (engineering) context of the given problem. Except privacy and use cases that are blocked by cloud models (e.g. reverse engineering), local LLMs are cur…

> currently

The interesting question is whether that gap will narrow, and if so, how much, and on what timescale.

The exact answer to this question is not knowable, but if you are the kind of person who comes to a site called "hacker news", and you think there is a nonzero chance that the answer is that yes, the gap will narrow and this won't always be an expensive toy, then now seems like a pretty great time to get in the game and start exploring the capabilities.

Re: Qwen 3.6 27B is the sweet spot for local development

#282

Post by stevenaenns »

iagooar wrote: post #274
xd1936 wrote: post #251

Earlier quoted context omitted.

Apple does not currently sell a Mac Mini with 64GB RAM.

Get a 2nd hand one. I was lucky enough to get a new one first, last week I get a 2nd hand one in order to run one of my Hermes minions at work.

how many tokens/s generation do you get?

Re: Qwen 3.6 27B is the sweet spot for local development

#283

Post by doodlesdev »

znpy wrote: post #193
doodlesdev wrote: post #83

I feel like I'm going insane seeing people buy these 128gb MBP for thousands of dollars to run models that are objectively much worse than SOTA and spending so much more. The amount spent on a 128gb M5 MAX can buy you a damned new car here. What the hell am I missing? Are developers in other countries living in such different worlds? (I'm aware the price is, in absolute terms, more expensive where I live compared to…

> Are developers in other countries living in such different worlds? Yes. Back in the my days at $faang in europe it was not uncommon to hear people getting 120-160 k€/year in compensation and we were “poor” compared to us engineers at the same faang (4-500 k$/year total compensation) with a bit of seniority…

That makes a lot of sense! I have no idea how I'd use that much money, so maybe the 128gb MBP for messing around with local LLMs wouldn't sound so absurd :)

Re: Qwen 3.6 27B is the sweet spot for local development

#284

Post by ricardobayes »

dofm wrote: post #50
bensyverson wrote: post #26

The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...

The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all th…

I'd say give it some time for the dust to settle. This field badly needs standardized benchmarks even before the conversation around model goodness can start.

Re: Qwen 3.6 27B is the sweet spot for local development

#285

Post by sanderjd »

wahnfrieden wrote: post #160
dofm wrote: post #141

Earlier quoted context omitted.

Again, I would not argue against any of this. And I can't say that I won't switch to openrouter (even just for the same models) at some point. But one of the things I have found about my own process learning is that some lessons only come to you when you make yourself available to them. And if that means doing things the difficult way, that is what you should do.

Difficult... and wastefully expensive

Seems like an investment into building expertise, which is likely to have high ROI in the future, rather than a wasteful cost.

Re: Qwen 3.6 27B is the sweet spot for local development

#286

Post by greyskull »

dofm wrote: post #119
ddalex wrote: post #81

Earlier quoted context omitted.

I just got Claude to download and install all the models and servers and agents and prepare all the launch scripts for me... no need to learn, just ask it to do it for you

Right, but I am a middle-aged bloke who is experiencing existential angst about whether I can carry on in this industry. I have a pretty deep, maybe paranoid need to be confident I have an intrinsic understanding, and I have found in my life that lessons come to you when you make yourself open to learning. So I need to build on top of what I know, taking as much of the hard way as I can bear to take at any one time —…

This really resonates with me, and I'm only a decade and change into my career. I use claude a lot day to day. I try to use it sensibly, making me more productive and produce better work. I'm also trying not to lose understanding along the way. I want to be able to actually talk to the conclusions I'm reaching.

I have colleagues that seem perfectly content to delegate too much to the agents, and it saddens me. It feels like there will be swaths of engineers that didn't train some of the critical thinking skills that I take for granted.

I certainly see it in slack discourse around anything more complicated than a feature implementation. Maybe I'm just cynical. Time will tell, I suppose.

Re: Qwen 3.6 27B is the sweet spot for local development

#287

Post by jarjoura »

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

TBF, I just recently picked up this same model, and it's reminding me of the last gen Intel i9 MBP. Just visiting any non-basic website spins up the fans and battery life isn't great either. Yes, this thing is fast, but damn it gets hot just using it for normal tasks.

Still, I don't agree. I think this machine is meant to use local models. You just have to wear pants if you want to keep it directly on your lap. I rarely use it that way anyway. I prefer it plugged into an external display and comfortably sitting on a laptop stand.

Re: Qwen 3.6 27B is the sweet spot for local development

#288

Post by hughw »

boutell wrote: post #123
stymaar wrote: post #79

Earlier quoted context omitted.

> The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Qwen3.6-27B would be faster on a 3090 that costs around $1000-1200 though so I don't think it's a good counter-argument. Op just happened to have that MacBook, but it doesn't mean it's necessary to run the model.

That 3090 is going to burn 750W and it will still cap you at a 4 bit quant and ~48K context. Here's someone who worked through it: https://github.com/noonghunna/qwen36-27b-single-3090 Flies though (50-70tps is impressive for a model this smart) I went through roughly the same process to get it working on my M2 Macbook Pro... at awful speeds of course, since models like this one are mostly bound by memory bandwidth.

My eyes glaze over reading all the AI produced verbiage.

I did find a few useful parameter settings I've already discovered using my single 3090 and ollama.

I'm just remarking that the LLMs overwhelm me with minutiae, especially as I'm working on code design. I frequently ask it to restate concisely, and that helps.

[edited to mention ollama as a nice alt]

Re: Qwen 3.6 27B is the sweet spot for local development

#289

Post by Fr0styMatt88 »

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

What kind of speed in tk/s do you get with the MacBook?

Re: Qwen 3.6 27B is the sweet spot for local development

#290

Post by iagooar »

seanmcdirmid wrote: post #277
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

What sort of M5 are you running? A max? MacMini's don't offer max CPUs.

M5 Max. But I also have a MacMini M4 Pro 64GB. Qwen3.6 runs on the M4 just fine - sure the M5 is at least 2x the speed. If Apple launches a MacMini with an M5, I will be the 1st one to get it.
Post reply on HN