[flagged]
Qwen 3.6 27B is the sweet spot for local development
601–610 of 809 posts
Re: Qwen 3.6 27B is the sweet spot for local development
#602>Real work
This part should have featured something about real work. But instead it features a paragraph about one-shot bs that creates "something".
Unless your work is to create thousands wordpress tremplates to sell - this is not a "real work".
Give it a repository (any kind of OSS project will do for an example) and a github issue requesting a knew feature or describing a confirmed bug. (you can and probably should write a prompt for LLM shough, don't just provide the issue itself)
And then whatch it go.
And then judge the result and it's quality.
Sorry, but from my experience 27B is just useless. You do get a result and some times it does work, but most of the times it is not event on junior dev level. And it takes it a lot of time to do the thing, unless you have an extremely expensive machine.
Re: Qwen 3.6 27B is the sweet spot for local development
#603qwen 3.6 27b and qen35b a3b work like magic, if we get dpark speculative decoding versions of these models it will further improve the throughput
Re: Qwen 3.6 27B is the sweet spot for local development
#604Earlier quoted context omitted.
Apple does not sell a 64GB variant of the M4 Mac Mini. IIRC they never have; its always capped out at 48GB. If you were planning on getting an M5 128GB; just get a DGX Spark (~$4500) or a 5090-equipped machine (~$4500) plus a Macbook Air (~$1500). You'll come in below the M5 Max 128 pricing (~$6700+ USD) and be happier for it.
DGX Spark everyone is saying performance for the money is not there
I have an access to a DGX spark, and while it performs better than my MacBook Pro (M3 Max), the performance on Qwen and Gemma dense models is dog shit, and not worth it.
Re: Qwen 3.6 27B is the sweet spot for local development
#605Earlier quoted context omitted.
The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all th…
> The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it. Seems like a GPU with 12GB+ VRAM is going to be a much more affordable way to achieve that? Even a B580 should get reasonable perf there.
No idea. I am a Mac guy, have been for a very long time. I buy them secondhand as a rule.
I guess I would build a powerful home LLM server if I was convinced I really needed one for my purposes for some agentic application or other. At the moment I'd prefer to ride this out with a machine that is also an excellent Mac.
Re: Qwen 3.6 27B is the sweet spot for local development
#606Earlier quoted context omitted.
(I'm not one of the people you're speaking of with a 128gb M5 but) if you want to run one of the medium-sized open-weights models (Qwen 27b, 35b, Gemma 4 26b, 31b) or larger, you get into an interesting optimisation space. * yes, you can run it on an older/smaller GPU plus system RAM but performance will suffer * if you want optimal GPU performance you need the model in VRAM plus context, so 24GB (3090, 4090) or 32GB…
And with a mac, there are no cuda drivers to fiddle with.
But prompt processing is terrible
Re: Qwen 3.6 27B is the sweet spot for local development
#607I've worked extensively with the slightly less able cousin, the 35B A3B model and tuned my own harness around making it work well with local or non-sota models. The results are quite promising [0], if one sticks to a plan-execute approach. After a bit of fiddling with llama.cpp I was able to get it to work through a small change on a real codebase from work on a 32GB M5 (typical python FastAPI backend, so nothing out of the ordinary). While that's somewhat encouraging, the whole local experience was still far from pleasant with all the noise and heat.
[0] https://deepclause.substack.com/p/how-to-make-small-models-p...
Re: Qwen 3.6 27B is the sweet spot for local development
#608Earlier quoted context omitted.
I second unsloth models. I'm using them over blackwell-oriented nvfp4 models as they are (empirically) top quality and performance.
NVFP4 will be better if the model provider actually post-trained properly after quantizing.
Which basically only Nvidia does, because it’s very expensive.
Though I’m currently working on QADing the smaller Qwen 3.5 models from FP16 teacher to NVFP4 student, to hopefully eventually apply it to 3.6 27B… harder to get right than I expected though!
Re: Qwen 3.6 27B is the sweet spot for local development
#609Has anyone considered a home server? Assuming mobility is not important if we pick components to match a similar hardware would it be more value for money?
I specifically chose a Mac Studio 128GB as my home server that's also running LLMs to be always online, in part due to the minimal idle power consumption and mostly fan-less operation. It's definitely expensive, especially nowadays, but I can still recommend Mac Minis as a cheaper alternative for someone to just get started with an affordable, always-on home server that won't annoy any housemates. I think both are in some sweet spot in terms of value for money, depending on what you're looking for in a home server. If image or video generation is your thing, look further though, definitely look into a proper GPU then. Macs are quite slow at that. They're just great at MoE LLMs because it's mostly a matter of (V)RAM size.
Re: Qwen 3.6 27B is the sweet spot for local development
#610[flagged]