Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

171–180 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#171

Post by dofm »

CamperBob2 wrote: post #159
coldtea wrote: post #147

Earlier quoted context omitted.

> no need to learn, just ask it to do it for you And that's how skills die.

When's the last time you shoed a horse? The reason I delegate so much of local LLM installation and administration to Claude Code is simply because there's no point learning practical things that will work completely differently in a couple of years, or in memorizing procedures that I'll forget long before I need to perform them again. No longer having to sweat all the details is a Good Thing, not a Bad Thing.

I am not sure I disagree, and I certainly don't mean to disagree very fervently.

But I think if you want to really learn to ride well, understand horses well, there might be some benefit in learning how to shoe a horse. At some level it should never only be someone else's job.

Re: Qwen 3.6 27B is the sweet spot for local development

#173

Post by prima-facie »

roadside_picnic wrote: post #62
petilon wrote: post #36

Earlier quoted context omitted.

Memory requirements of newer models will increase, so while the hardware may last 10 years it won't be able to run the latest models for 10 years.

My experience working in the open model space pretty deeply (both LLMs and diffusion models) for years now is that it is not quite as simple as that. In the open model space an insane amount of effort goes into getting more powerful models to run with the same or less RAM. For example in the diffusion world many things that could not be run on easily under 24GB of VRAM actually run much better today with much less VR…

> The biggest thing to watch out for is not just RAM/VRAM but memory bandwidth. You can try to "future proof" yourself with lots of RAM, but if it's 400 GB/S you're still constrained to smaller models.

I'm thinking of getting a SoC machine with 128GB RAM but the bandwidth is limited to 256 GBps. Would you even consider such a machine a decent investment, or should I wait for the newer gen of chips? Thanks!

Re: Qwen 3.6 27B is the sweet spot for local development

#174

Post by Otternonsenz »

fumeux_fume wrote: post #115
Otternonsenz wrote: post #91

Is there any hope for people that cant even run 27B parameters, Qwen3.6 or otherwise? Are there any quantized models that do well with tool calling at smaller parameter sizes? I do not have a crazy rig, a modest gaming one at that, but in trying to understand more about agents and their capabilities, I am SOL with my 16 GB of RAM and 8GB of VRAM. I can get most small, non tool calling models to perform well, but I've…

I suspect with those specs, you're not in the game right now for reliably using local models for code generation. The easiest way in is a MacBook with at least 32GB of RAM. This should be able to run a 4bit quantization of qwen 3.6 using the MLX format really well.

Now that I’m dipping more into this space, am gonna see what I can upgrade with the motherboard I have, but RAM pricing as it is, I’ll need to be smart about when I upgrade.

I very much appreciate the frank response, as it makes me feel less defeated at knowing my understanding of how it should work is not the full issue, hahaha

Re: Qwen 3.6 27B is the sweet spot for local development

#175

Post by DANmode »

wilsonnb3 wrote: post #164
throw1234567891 wrote: post #125

Earlier quoted context omitted.

But the tokens or credits are gone. MacBook stays. You can run other models on the same MacBook. What I read people burn every month on saas… for that money you break even on that MacBook in 5 months. Edit: it’s not just “data privacy”, when you are using Claude, you are shipping EVERYTHING to Anthropic. It’s crazy.

Companies are already shipping everything to Microsoft or Google and 17 other companies, just the cost of doing business.

That’s at today-prices.

If the cost doubles, or 4x, which is seems to need to for them to go profitable, what then?

Re: Qwen 3.6 27B is the sweet spot for local development

#176

Post by psychoslave »

CamperBob2 wrote: post #159
coldtea wrote: post #147

Earlier quoted context omitted.

> no need to learn, just ask it to do it for you And that's how skills die.

When's the last time you shoed a horse? The reason I delegate so much of local LLM installation and administration to Claude Code is simply because there's no point learning practical things that will work completely differently in a couple of years, or in memorizing procedures that I'll forget long before I need to perform them again. No longer having to sweat all the details is a Good Thing, not a Bad Thing.

Having to shoe a horse never was a general skill.

Maybe a more apt analogy would be a skill like making fire without a lighter.

Re: Qwen 3.6 27B is the sweet spot for local development

#177

Post by kmike84 »

mannyv wrote: post #172

FYI token speed is somewhat irrelevant for agentic development. You let it run, then you come back. The whole point is that it's asynchronous. If it takes 4 hours, 8 hours, 16 hours...who cares?

You care if you run it on a laptop. It's getting hot, fans are spinning, and you may want to use laptop for other things while the agent is working.

Re: Qwen 3.6 27B is the sweet spot for local development

#178

Post by snapcaster »

h4ny wrote: post #52
onion2k wrote: post #20

None of the examples reflect 'real work', at least not what I'd consider real work. Being able to nail a zero-shot greenfield project is relatively easy even for a small model. There's not much context to build up and it can fall back to similar examples in the training data easily. So long as you're not asking it to invent something wholly new it'll probably manage. The real test is whether or not it can work with y…

> In my limited experiments Qwen 3.5 (maybe 3.6 is loads better) 1. Maybe you should tell us what those limited experiments are. 2. Maybe you should actually try 3.6 because it's huge difference in most cases. Don't forget to tell us quants and don't forget to tell us scope. 3. Maybe actually show us data compared to frontier models instead of this... vibe comment. Pretty tired of this kind of comments on HN that doe…

Nobody owes you a scientifically rigorous write up

Re: Qwen 3.6 27B is the sweet spot for local development

#179

Post by swiftcoder »

ddalex wrote: post #81
dofm wrote: post #50

Earlier quoted context omitted.

The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all th…

I just got Claude to download and install all the models and servers and agents and prepare all the launch scripts for me... no need to learn, just ask it to do it for you

I don't necessarily think your answer is wrong for all people, but if you work in software... how do you plan to differentiate yourself from everyone else out there, if the depth of your understanding is "Claude can do it for me"?

Re: Qwen 3.6 27B is the sweet spot for local development

#180

Post by Otternonsenz »

fluoridation wrote: post #140
Otternonsenz wrote: post #91

Is there any hope for people that cant even run 27B parameters, Qwen3.6 or otherwise? Are there any quantized models that do well with tool calling at smaller parameter sizes? I do not have a crazy rig, a modest gaming one at that, but in trying to understand more about agents and their capabilities, I am SOL with my 16 GB of RAM and 8GB of VRAM. I can get most small, non tool calling models to perform well, but I've…

I think at 16 GB you'd struggle to run the regular development tools nowadays, forget about any interesting inference.

Fully agreed, and my hope is as open models grow and change, that getting some amount of this working on Pro-sumer hardware will be more attainable.

But certainly seems like we are a few years away from that, sadly.

Am I also screwed in being able to train my own small model or adjust another one with such a non-workhorse PC?

Post reply on HN