It's a rap on claude
Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
221–230 of 482 posts
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#222Has anyone tried using this with a Claude Code or Qwen Code? They both require very large context windows (32k and 16k respectively), which on a Mac M4 48GB serving the model via LM Studio is painfully slow.
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#223Earlier quoted context omitted.
I get ~5 tokens/s on an M4 with 32G of RAM, using: llama-server \ -hf unsloth/Qwen3.6-27B-GGUF:Q4_K_M \ --no-mmproj \ --fit on \ -np 1 \ -c 65536 \ --cache-ram 4096 -ctxcp 2 \ --jinja \ --temp 0.6 \ --top-p 0.95 \ --top-k 20 \ --min-p 0.0 \ --presence-penalty 0.0 \ --repeat-penalty 1.0 \ --reasoning on \ --chat-template-kwargs '{"preserve_thinking": true}' 35B-A3B model is at ~25 t/s. For comparison, on an A100 (~RTX…
When you say tok/s here are you describing the prefill (prompt eval) token/s or the output generation tok/s? (Btw I believe the "--jinja" flag is by default true since sometime late 2025, so not needed anymore)
Here is llama-bench on the same M4:
| model | size | params | backend | threads | test | t/s |
| ------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: |
| qwen35 27B Q4_K_M | 15.65 GiB | 26.90 B | BLAS,MTL | 4 | pp512 | 61.31 ± 0.79 |
| qwen35 27B Q4_K_M | 15.65 GiB | 26.90 B | BLAS,MTL | 4 | tg128 | 5.52 ± 0.08 |
| qwen35moe 35B.A3B Q3_K_M | 15.45 GiB | 34.66 B | BLAS,MTL | 4 | pp512 | 385.54 ± 2.70 |
| qwen35moe 35B.A3B Q3_K_M | 15.45 GiB | 34.66 B | BLAS,MTL | 4 | tg128 | 26.75 ± 0.02 |
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#224Does anyone know good provider for low latency llm api provider? We tried to look at Cerebras and Groq but they have 0 capacity right now. GPT models are too slow for us at the moment. Gemini are better but not really at same level as GPT.
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#225Been using Qwen 3.6 35B and Gemma 4 26B on my M4 MBP, and while it’s no Opus, it does 95% of what I need which is already crazy since everything runs fully local.
You've got me curious. Two questions if I may: - What kind of tasks/work? - How is either Qwen/Gemma wired up (e.g. which harness/how are they accessed)? Or to phase another way; what does your workflow/software stack look like?
1. Qwen is mostly coding related through Opencode. I have been thinking about using pi agent and see if that works better for general use case. The usefulness of *claw has been limited for me. Gemma is through the chat interface with lmstudio. I use it for pretty much everything general purpose. Help me correct my grammar, read documents (lmstudio has a built in RAG tool), and vision capabilities (mentioned below, journal pictures to markdown).
2. Lmstudio on my MacBook mainly. You can turn on an OpenAI API compatible endpoint in the settings. Lmstudio also has a headless server called lms. Personally, I find it way better than Ollama since lmstudio uses llama cpp as the backend. With an OpenAI API compatible endpoint, you can use any tool/agent that supports openAI. Lmstudio/lms is Linux compatible too so you can run it on a strix halo desktop and the like.
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#226Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#227I wish that all announcements of models would show what (consumer) hardware you can run this on today, costs and tok/s.
I'm getting 30 t/s on RTX 4090D (using 42 out of 48GB VRAM) with UD-Q6_K_XL
https://huggingface.co/unsloth/Qwen3.6-27B-GGUF/discussions/...
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#228Has anyone tried using this with a Claude Code or Qwen Code? They both require very large context windows (32k and 16k respectively), which on a Mac M4 48GB serving the model via LM Studio is painfully slow.
I had the best success yet earlier today running https://pi.dev with a local gemma4 model on ollama on my m4 Mac with 48GB ram. I think pi is a lot lighter than Claude code.
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#229I'll be really interested to hear qualitative reports of how this model works out in practice. I just can't believe that a model this small is actually as good as Opus, which is rumored to be about two orders of magnitude larger.
Re: Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model
#230I wish that all announcements of models would show what (consumer) hardware you can run this on today, costs and tok/s.
I'm getting 30 t/s on RTX 4090D (using 42 out of 48GB VRAM) with UD-Q6_K_XL https://huggingface.co/unsloth/Qwen3.6-27B-GGUF/discussions/...
I thought Q4_K_M is the standard. Why did you choose the 6-bit variant? Does it generate better input?