Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

701–710 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#701

Post by regularfry »

petilon wrote: post #36
Insanity wrote: post #30

Earlier quoted context omitted.

But you have to factor in that this device will last you 5-10 years. That said, I wouldn't spend almost $7k USD on this macbook lol.

Memory requirements of newer models will increase, so while the hardware may last 10 years it won't be able to run the latest models for 10 years.

Nah. There are already models at every size on the scale. If you want to run an open 1T model today, you can.

What's going to happen is that the capability at any given size point is going to get better over time as new training regimes cram more into the available space. A 27b model released next year will be better than a 27b model this year (else why release it?). Hardware will get more useful, not less.

Re: Qwen 3.6 27B is the sweet spot for local development

#702

Post by musebox35 »

dofm wrote: post #50
bensyverson wrote: post #26

The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...

The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all th…

Thanks for posting this. This is the tinkerer mentality. It is not for everyone, but certain things can only be learned in that way. It is the best antidote to AI paranoia. There is much that does not transfer between frontier models and local ones. There is that. But you can not tinker as much as you can with the former.

Re: Qwen 3.6 27B is the sweet spot for local development

#703

Post by __MatrixMan__ »

v1ne wrote: post #623
alemanek wrote: post #564

Earlier quoted context omitted.

If your workflow benefits from the speed it quickly pays for itself when factoring in developer salaries here in the US. I recently switched companies and they bought me an M5 Max 128GB as my dev machine. Builds and local test runs are 3 times faster than the Windows laptop option. The machine will pay for itself just based on that within 3 months. I can spin up a local kubernetes cluster and do full integration test…

Don't worry. Once IT Security discovers that they miss their trusty endpoint security products on your Mac, they'll add it and you'll be in the same ballpark as the Windows machine. Been there, received that, and learnt that Microsoft Defender exists for macOS, too.

That's why you gotta have your work laptop, and your "work" laptop. Give them something to secure. It makes them feel important.

Re: Qwen 3.6 27B is the sweet spot for local development

#705

Post by sanderjd »

nazgul17 wrote: post #668
sanderjd wrote: post #336

Earlier quoted context omitted.

To me, this is just a question of specialization. Not everyone needs to be a "I understand how the system actually works" person. In fact, not many people need to be that person. But every system does need some of that person to exist! And we happen to be discussing this on a forum where the type of people who will be the specialists for the kinda of systems we're discussing are likely to gather. I'd be surprised if…

I think the more you know of how (many) things work, the slightly better you'll be at using them. From dishwashers to CPUs, from car engines to watercolours, from guitars to kitchen knives... You get the gist. Once you internalize a model of the thing, it becomes closer to an extension of you than a tool. You drive it better and with less friction.

Yes agreed, but there is limited time in a life, so there is a fairly high opportunity cost to internalizing a model of many things, which scales quickly with the complexity of those things, so people rationally limit the number of things they invest their time in. For the vast majority of people, I think it makes a lot of sense for AI systems to fail to make this cut. But for most of us here, on a site for computer technologists, it almost certainly makes sense for us to learn as many of the details as we can manage.

Re: Qwen 3.6 27B is the sweet spot for local development

#706

Post by Roark66 »

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

I think there is no reasonably priced machine you could run locally to do serious work with LLMs...

10x rtx6000 Pro in a large workstation is probably the way to go for someone wanting to run GLM5.2.

Other than that it is cloud.

As good as these small models got we are still not "at breakeven" for me.

What is "breakeven" with LLMs? For me it is when I no longer have to read the actual code it wrote. I can trust that if I told it to implement and document a certain architecture it actually did that with no stupid mistakes.

The first model ever that did that for me was the first opus. 4.4 if I remember correctly.

The second model was Gemini 3 Pro preview. For few weeks. Then it was lobotomised. I guess it was too expensive to run and they quantized it too hell.

Only Opus remains. If this GLM model truly rivals even an old opus I'll be very happy when day comes that I'll be able to run it locally.

Re: Qwen 3.6 27B is the sweet spot for local development

#707

Post by jarek83 »

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

You can't buy Mac Mini with 64GB RAM today. Most what you can have is 48GB

Re: Qwen 3.6 27B is the sweet spot for local development

#708

Post by Roark66 »

bilekas wrote: post #451
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

Can you define "serious programming"? Because I use it to implement things I COULD go and figure out like algorithms or test generation or evaluations etc, the "serious" programming I tend to do myself. That is what I'm paid for.

Serious programming is dealing with a large knowledge surface area.

So not "implement me a shading algorithm"

But more like: make an multi user app running on a k8 cluster, design the whole thing to be indempotent, scalable, easy to deploy remotely via ipmi/pxe boot.

Then see how it makes stupid mistakes along the way.

Today's AI is pretty amazing when it comes to fixing narrow problems (or creating Web apps with no infra). Give it anything where it needs to go online, download some helm templates and look through them to figure out parameters, as well as write an app and it will make lots of mistakes in seemingly simple stuff.

Opus seems to be the model that works the best with this.

Re: Qwen 3.6 27B is the sweet spot for local development

#710

Post by Roark66 »

iagooar wrote: post #410
ActorNightly wrote: post #346

Earlier quoted context omitted.

>If you want to run Qwen3.6 27B / 35B at its best, get a MacMini M4 with 64GB of RAM and put it in the basement Im sorry, but its time to start calling Apple sycophants out. Stop trying to push your tech jewelry on other people. You only buy those computers because they are Apple, you don't know anything about computing or running LLMs, you don't do any real work, so you should probably not give advice on what to buy…

I am not going to flag you, I am much OK with having good arguments. I just purchased a Mac Mini M4 Pro 64GB for $3k - 2nd hand of course. I am not a hater of Nvidia and I am planning on building a workstation based on RTX cards. You clearly do not seem to understand how convenient the MacMini actually IS - the form factor, how quiet it is, how durable it is, how well it integrates with other Macs, how well it works…

I have to see I'm in the rtx camp. A dual rtx3090 workstation with 200G of ram and zen5 9950x cpu. All watercooled.

The only reason I can tell it's on, is the very quiet hum of the slow speed water pump. Large fans run at 1200rpm and are fully quiet.

I have over a meter of radiators there.

Fun fact, I bought my first rtx3090 4 years ago. A year ago I bought another one and they are still the same price used.

I may buy another one (for my servers)

Post reply on HN