Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.6 27B is the sweet spot for local development

quesma.com

Post reply on HN
Print view View original

621–630 of 809 posts

Re: Qwen 3.6 27B is the sweet spot for local development

#621

Post by alentred »

mashygpig wrote: post #455

It's fun to run a model locally, but I don't think the economics make sense for anyone just trying to use models atm. It's absurdly cheap to use the same model via openrouter in comparison. Seriously, just put $10 into openrouter and play with models that are cheap but bigger than what you'd reasonably be able to run locally like deepseek v4 flash (unquantized). You'll be surprised by how far that $10 goes for a mode…

There is one side effect of running your LLM locally: you stop thinking about the token budget. I often run `/goal` with no limits, or script an endless loop in bash to run opencode, etc. Sometimes I just brute force the task by throwing a /goal at it. Maybe it's not the most efficient use either, but it's nice to have the option.

Re: Qwen 3.6 27B is the sweet spot for local development

#622

Post by pistoriusp »

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

Mac Mini in the rack and a Neo in the lap.

Re: Qwen 3.6 27B is the sweet spot for local development

#623

Post by v1ne »

alemanek wrote: post #564
doodlesdev wrote: post #83

I feel like I'm going insane seeing people buy these 128gb MBP for thousands of dollars to run models that are objectively much worse than SOTA and spending so much more. The amount spent on a 128gb M5 MAX can buy you a damned new car here. What the hell am I missing? Are developers in other countries living in such different worlds? (I'm aware the price is, in absolute terms, more expensive where I live compared to…

If your workflow benefits from the speed it quickly pays for itself when factoring in developer salaries here in the US. I recently switched companies and they bought me an M5 Max 128GB as my dev machine. Builds and local test runs are 3 times faster than the Windows laptop option. The machine will pay for itself just based on that within 3 months. I can spin up a local kubernetes cluster and do full integration test…

Don't worry. Once IT Security discovers that they miss their trusty endpoint security products on your Mac, they'll add it and you'll be in the same ballpark as the Windows machine. Been there, received that, and learnt that Microsoft Defender exists for macOS, too.

Re: Qwen 3.6 27B is the sweet spot for local development

#624

Post by iagooar »

astrostl wrote: post #474
iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

> MacBook Pro M5 128GB RAM 614 GB/s of memory bandwidth > MacMini M4 with 64GB of RAM 273 GB/s of memory bandwidth (also only currently available with 48GB) When it comes to inference speed, you want your model to fit in memory, and then to have as much memory bandwidth as possible. In this case a hypothetical Mini with 1TB of memory would still be over 2x slower with 27-35B models. And FWIW I have an M4 Max MBP 128G…

On paper the M4 should be roughly 1/3 of the M5, in practice it is only 1/2. With the right, optimized model like qwen3.6 35B MoE MLX you can get over 40 tok / sec on it. I run dozens of background jobs that are not time-critical on it.

Re: Qwen 3.6 27B is the sweet spot for local development

#625

Post by PeterStuer »

iagooar wrote: post #199

I love my MacBook Pro M5 128GB RAM and I love qwen3.6. BUT DO NOT buy this MacBook if you plan on doing serious coding using local LLMs with it. The reason is simple: your fingers will burn and your head will explode from the noise. Running any kind of sophisticated job on the very laptop you are using is just not viable. Sure you can use it in clamshell mode, but forget touching it while working with AI coding or ag…

No laptop is thermally designed to handle sustained high workloads. The whole point of a laptop is to keep it thin, quiet and light, the exact opposite of what cooling needs.

Re: Qwen 3.6 27B is the sweet spot for local development

#627

Post by nunodonato »

imrehg wrote: post #612

I'm having a decently good time time with `qwen3.6-35b-a3b-mtp` (unsloth's multi-token prediction version) and and `qwen-agentworld-35b-a3b`. On a 2021 M1 Pro (32GB RAM) I can get either of them as `IQ4_NL` quantized models (the first with reduced context, around 160k; the second can do the whole 264k with RAM left over), running something like 30tokens/s. On a Framework 13 AMD AI HX370 it can use the same, but both…

what are you using agentworld for?

Re: Qwen 3.6 27B is the sweet spot for local development

#628

Post by Gigachad »

XCSme wrote: post #424

Considering the cloud version, all three models compared in the article (Qwen 3.6 35BA3b, 3.6 27B and DeepSeek V4 Flash), have very similar performance[0], BUT on cloud, for some reason DeepSeek V4 Flash is 10-20x cheaper than the Qwen models. If Qwen models are so much easier to run, why are the providers charging more than V4 Flash? [0]: https://aibenchy.com/compare/qwen-qwen3-6-35b-a3b-medium/qwe... <-- compare ho…

Also confused by this. Deepseek V4 flash is so much better than Qwen 3.6 yet cheaper to use.

Re: Qwen 3.6 27B is the sweet spot for local development

#629

Post by hypfer »

konart wrote: post #602

>Real work This part should have featured something about real work. But instead it features a paragraph about one-shot bs that creates "something". Unless your work is to create thousands wordpress tremplates to sell - this is not a "real work". Give it a repository (any kind of OSS project will do for an example) and a github issue requesting a knew feature or describing a confirmed bug. (you can and probably shoul…

If your expectation is to treat it as a coworker, then you're right.

If your expectation is to treat it as a tool, then you're wrong.

I guess that's where the disconnect lies.

Re: Qwen 3.6 27B is the sweet spot for local development

#630

Post by hollowturtle »

spiralcoaster wrote: post #427
hollowturtle wrote: post #387

Earlier quoted context omitted.

In what era spinning up a PoC required a week of work? Especially on the web. I've been a developer for roughly 20 years and that has never been the case, to the point that I believe people impressed by LLMs are the same who had a very low productivity. Today we have game jams as short as 3 days and talented people are able to produce very good PoC, with some almost complete!

[flagged]

These people work mostly in CRUD apps and they're telling you they how feel productive. Btw exploratory ideas even for hard problems come out already after a hackaon of a day or a game jam of 3 days
Post reply on HN