I've come from the future to say Qwen 3.7 27B is just around the corner and slaps!
I am eagerly waiting!
Qwen 3.6 27B is the sweet spot for local development
11–20 of 809 posts
Re: Qwen 3.6 27B is the sweet spot for local development
#12This is kind of like saying grass is green to be honest
Like everybody got 128 GB RAM..
Re: Qwen 3.6 27B is the sweet spot for local development
#13How does llama.cpp use the GPU efficiently as opposed to MLX?
Is there any way to use MLX and GPU at the same time? Or does memory become a big problem?
TBH, I never understood Apple hyping these neural cores because I didn't think anyone actually uses them except maybe certain photo/video editing software.
If I can generate voice at the same time as video, that would be useful.
Re: Qwen 3.6 27B is the sweet spot for local development
#14I have been having pretty good success with Qwen 3.5 9B for "nontrivial but not challenging work all things considered" -- it runs great on my 24gb unified memory m4 pro MacBook Pro. What do the baseline specs look like Mac-wise for getting this model to run? Am I looking at a 96gb? 128? 256?
You might be interested in Ornith 1.0 9B, which is a new intriguing post-training of Qwen 3.5 9B.
Qwen 3.6 27B will run in full offload with a 4-bit quantisation in 64GB on an M1 Max. It is quite slow.
I don't know about 48GB but 64GB should be enough.
Re: Qwen 3.6 27B is the sweet spot for local development
#15Re: Qwen 3.6 27B is the sweet spot for local development
#16> What it does:
>
> --jinja for tool calling support
Pretty sure this flag hasn't done anything for a while. It's enabled by default since ~November of last year
Re: Qwen 3.6 27B is the sweet spot for local development
#17I have been having pretty good success with Qwen 3.5 9B for "nontrivial but not challenging work all things considered" -- it runs great on my 24gb unified memory m4 pro MacBook Pro. What do the baseline specs look like Mac-wise for getting this model to run? Am I looking at a 96gb? 128? 256?
You might be interested in Ornith 1.0 9B, which is a new intriguing post-training of Qwen 3.5 9B. Qwen 3.6 27B will run in full offload with a 4-bit quantisation in 64GB on an M1 Max. It is quite slow. I don't know about 48GB but 64GB should be enough.
Thanks! I was thinking of doing the 128gb to have some future proofing. I figure at this point, it's akin to a mechanic keeping great tools around, when it comes to having this sort of homelab and exposing it for your own uses. And great practice for building the next era of user facing computing that will be around as this proliferates.
Re: Qwen 3.6 27B is the sweet spot for local development
#18Very capable lora adapters are surfacing but it seems they are very niche.
Re: Qwen 3.6 27B is the sweet spot for local development
#19> ... on my Macbook Max M5 128 GB
Local development for who? How many of y'all are rocking 128GB of memory? Am I reading Apple's site correctly that it's a $10,000 laptop?
Re: Qwen 3.6 27B is the sweet spot for local development
#20None of the examples reflect 'real work', at least not what I'd consider real work. Being able to nail a zero-shot greenfield project is relatively easy even for a small model. There's not much context to build up and it can fall back to similar examples in the training data easily. So long as you're not asking it to invent something wholly new it'll probably manage.
The real test is whether or not it can work with your existing codebases. In my limited experiments Qwen 3.5 (maybe 3.6 is loads better) does OK on a Rust+React app, and less well on a C# monolith. Not to the point of being unusable but definitely poorly enough that I went back to Claude after 20 minutes. If I lost access to a cloud model and had to use Qwen instead I'd be visibly sad.