Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

581–590 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#581

Post by internet101010 »

onion2k wrote: post #20

None of the examples reflect 'real work', at least not what I'd consider real work. Being able to nail a zero-shot greenfield project is relatively easy even for a small model. There's not much context to build up and it can fall back to similar examples in the training data easily. So long as you're not asking it to invent something wholly new it'll probably manage. The real test is whether or not it can work with y…

Exactly. If the repo has all of the knowledge living inside of it that window fills up fast, even when using something like codegraph.

Re: Qwen 3.6 27B is the sweet spot for local development

#582

Post by QuercusMax »

dofm wrote: post #404
QuercusMax wrote: post #343

Earlier quoted context omitted.

Yeah, it's been fun for me running models (mostly Qwen 3.6 27B) on my 48GB M4 MacBook Pro. When i'm using it to run models, it's basically unusable for anything else - I actually do the work on my Macbook Neo. Took me a while to figure out why the models couldn't figure out how to make tool calls - because LMStudio by default uses a 32K input window, which is smaller than OpenCode's prompt, so half of the instruction…

Yes — there is a setting for that isn't there. And as soon as you realise there's a setting for that, you have new knowledge. Qwen barely needs any of Opencode's prompt, in my experience; I think I cut it down to about three general lines I found by googling. Mainly you need only a pre-amble to make sure that the plan mode, plan switch and build mode prompt fragments make sense. Gemma 4 also needs almost nothing at a…

[dead]

Re: Qwen 3.6 27B is the sweet spot for local development

#584

Post by anon373839 »

jasonjmcghee wrote: post #579
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

I'm surprised no one has else has mentioned - low power mode. With no speculative decoding, using high power mode, I get 80 t/s on 35B A3B - and it gets hot and spins up. On low power mode I get 38 t/s - no fans, cool to warm laptop. If you currently don't use speculative decoding and you start using it, it can nearly offset the difference between high and low power, and it's night and day experience. I almost always…

Can you mention what inference stack you're using? I've tried MTP several times with that model and it always seems to significantly cut my token generation speed from ~60 tokens/sec to ~40 (M3 Max).

Re: Qwen 3.6 27B is the sweet spot for local development

#585

Post by rvz »

When reading the comments, it seems that in the AI race to zero, Apple was already at the finish line. as predicted.

So it will be no surprise that there will be a time where everyone will be able to run a local model, say GLM 5.2 locally on their machine. Like it or not.

Re: Qwen 3.6 27B is the sweet spot for local development

#586

Post by sixothree »

amatecha wrote: post #543
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

I wonder if that's why there is such a good selection of 128gb M5 MBP's on the Apple Certified Refurbished store lol https://www.apple.com/ca/shop/refurbished/mac/macbook-pro-12...

Wait. Did they raise their prices a second time?

Re: Qwen 3.6 27B is the sweet spot for local development

#587

Post by sixdimensional »

Dig1t wrote: post #112
georgeven wrote: post #58

Earlier quoted context omitted.

I have a 1500 dollar machine that can run it at 50 tok/s (3 V100s)

How did you buy 3 V100's for $1500??

Not OP and just guessing, but probably SXM2 GPU modules for the V100. Those can be acquired fairly inexpensively, but there is work to do to get them working together and the V100 has some limitations on the types of models you can run.

Re: Qwen 3.6 27B is the sweet spot for local development

#588

Post by nirvdrum »

sixothree wrote: post #586
amatecha wrote: post #543

Earlier quoted context omitted.

I wonder if that's why there is such a good selection of 128gb M5 MBP's on the Apple Certified Refurbished store lol https://www.apple.com/ca/shop/refurbished/mac/macbook-pro-12...

Wait. Did they raise their prices a second time?

Probably USD vs CAD. The parent posted a /ca/ link, which will look really similar to /us/, but the prices will all appear to be higher.

Re: Qwen 3.6 27B is the sweet spot for local development

#589

Post by ozim »

827a wrote: post #518
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

Apple does not sell a 64GB variant of the M4 Mac Mini. IIRC they never have; its always capped out at 48GB. If you were planning on getting an M5 128GB; just get a DGX Spark (~$4500) or a 5090-equipped machine (~$4500) plus a Macbook Air (~$1500). You'll come in below the M5 Max 128 pricing (~$6700+ USD) and be happier for it.

DGX Spark everyone is saying performance for the money is not there
Post reply on HN