Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

371–380 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#371

Post by greyskull »

sanderjd wrote: post #339
dofm wrote: post #119

Earlier quoted context omitted.

Right, but I am a middle-aged bloke who is experiencing existential angst about whether I can carry on in this industry. I have a pretty deep, maybe paranoid need to be confident I have an intrinsic understanding, and I have found in my life that lessons come to you when you make yourself open to learning. So I need to build on top of what I know, taking as much of the hard way as I can bear to take at any one time —…

Yeah, this is just the engineer's mindset. It's not surprising that this is a popular view here, even if it is not (and does not need to be) the mainstream perspective.

> mainstream

What does "mainstream" refer to when we're talking about software development and LLMs? As opposed to "engineers".

Re: Qwen 3.6 27B is the sweet spot for local development

#372

Post by rnxrx »

lee_ars wrote: post #323
acters wrote: post #212

Earlier quoted context omitted.

Would the new upcoming AMD AI ryzen halo desktop be a better value offer? or dgx spark? You would have to get a third party reseller/scalper or refurbished mac mini to get 64gb of ram ever since apple stopped selling it.

I'm currently fiddling with a DGX Spark and Qwen3.6-35B-A3B (specifically Qwen3.6-35B-A3B-NVFP4 under vLLM, with EAGLE3 speculative decoding via eagle3-dogacel-vllm), and it's pretty okay in terms of smarts. The speed is relatively usable at about 50 tok/sec with a 256k context window, and it's definitely smart enough to one-shot some basic coding tasks. I had it doing reverse engineering/disassembly of some ancient…

There are also nvfp4 quants of Qwen 3.6 27/35 floating around. I've done benchmarks of both and the quality difference vs fp8/bf16 was barely notable. Honestly the nvfp4 capability is the most interesting feature of the Spark (at least for me).

Re: Qwen 3.6 27B is the sweet spot for local development

#373

Post by dang »

h4ny wrote: post #37
bensyverson wrote: post #26

The article is based on running Qwen 3.6 on a 128GB MacBook Pro. For reference, a 128GB MBP currently starts at $6699 USD [0] Some people will be happy to pay that premium for privacy, but at roughly 10X the cost of a MacBook Neo, that money could also buy a lot of credits on OpenRouter or frontier labs. [0]: https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...

[flagged]

Yikes, you broke the site guidelines badly with this post. Could you please review https://news.ycombinator.com/newsguidelines.html and stick to them?

You're welcome to make your substantive points thoughtfully, just not aggressively.

Re: Qwen 3.6 27B is the sweet spot for local development

#376

Post by aurareturn »

codazoda wrote: post #367
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

Today the Mini tops out at 48GB. Gotta go to the Studio to get 64GB.

Don't buy the Mini or Studio. Both have the M4 which lacks the Neural Accelerators, making prompt processing ~3-4x slower.

Re: Qwen 3.6 27B is the sweet spot for local development

#378

Post by sanderjd »

greyskull wrote: post #362
sanderjd wrote: post #344

Earlier quoted context omitted.

For me (about halfway between you and dofm in my career by your own statements in this thread), it's a dream at the moment. I can delegate all the tedious stuff that I've done "the hard way" a thousand times already and feel I have very little of value remaining to learn, so that I can spend more time on all the things that are actually new and thus much more interesting.

It's been a great multiplier for me in similar ways. The "dreamiest" thing has been that it has freed up time that I would normally have spent doing sprint work, to work on things that just don't make the cut until it's bad enough to deprioritize other work. Over the last few months, I've been digging into performance problems with a high throughput service that my team owns. I started working on the problems in my o…

Yes exactly. There is a narrative that it's driving everything toward low quality slop, but in my own work it's exactly the opposite. We're doing work on quality and performance that we never would have gotten to in the past.

I've spent my whole career being frustrated by the pile of low severity bugs and performance issues that "I could fix that if I could only justify putting a couple hours into it!". And now I can just fix all those. Nobody is going to question my use of time to write prompts and do code reviews of those things, when I can to my "real" work simultaneously.

Re: Qwen 3.6 27B is the sweet spot for local development

#379

Post by c7b »

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

This. Do consider local LLMs, but set aside a dedicated machine for it. Connect via VPN or reverse proxy. If it's not a Mac them I'd also put a server distro on it. No need for a desktop environment, save your RAM.

Re: Qwen 3.6 27B is the sweet spot for local development

#380

Post by simplyluke »

reilly3000 wrote: post #316
doodlesdev wrote: post #83

I feel like I'm going insane seeing people buy these 128gb MBP for thousands of dollars to run models that are objectively much worse than SOTA and spending so much more. The amount spent on a 128gb M5 MAX can buy you a damned new car here. What the hell am I missing? Are developers in other countries living in such different worlds? (I'm aware the price is, in absolute terms, more expensive where I live compared to…

It’s an asset on my balance sheet that’s already appreciating nicely and will likely be resale-able for what I paid for it for the next 7-10 years. I am on an Apple monthly installment plan so $5k is $416/month for 1 year, no interest. I’m able to run DS4 scale models and other open models without quantization, often multiple at once. Imagine its value if war broke out over Taiwan / Greater China, or really any of th…

> will likely be resale-able for what I paid for it for the next 7-10 years

There is not a period in the history of computing where this is true of consumer hardware over a decade for anything other than hardware already at the very bottom of its depreciation curve. It is surprising to me that you state that as an obvious assumption.

I suppose if your base case is Taiwan war that may be true, but there's a lot of folks who seem to be assuming the current hardware crunch will go on indefinitely when the natural state of hardware is getting cheaper over time.

Post reply on HN