Qwen3.7-Max: The Agent Frontier

Post by pixelesque » Wed, May 20, 2026, 6:24 PM UTC

gcr wrote: post #160

pixelesque wrote: post #150

Earlier quoted context omitted.

Out of interest, what machine and model are you running it on? I tried the qwen3.6-27b Q6_k GUFF in llama.cpp and LM Studio on my M2 MacBook Pro 32GB machine last week, and I barely get a token a second with either. What sort of speed should I be expecting? I tried some of the Llama 3 34b (nous-capybara?) models two years ago with llama.cpp, and I seem to remember getting a few tokens a second then, so not sure if I'…

There are two flavors of Qwen 3.6: - A 27B "dense" model - A 35B "Mixture of Experts" model, which activates only 3B parameters for each token. For your hardware, I strongly recommend `unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M`. I have an M1 Max with 32GB VRAM from 2021 that can read at ~300-500 tokens/sec and write at ~30 tokens/sec with llama-cpp's default settings, which is plenty fast. The 27B model can read ~70tok/sec…

Post by pixelesque » Wed, May 20, 2026, 6:25 PM UTC

Figs wrote: post #158

pixelesque wrote: post #150

Earlier quoted context omitted.

Out of interest, what machine and model are you running it on? I tried the qwen3.6-27b Q6_k GUFF in llama.cpp and LM Studio on my M2 MacBook Pro 32GB machine last week, and I barely get a token a second with either. What sort of speed should I be expecting? I tried some of the Llama 3 34b (nous-capybara?) models two years ago with llama.cpp, and I seem to remember getting a few tokens a second then, so not sure if I'…

27B is the dense one. Try the Qwen3.6-35B-A3B variants for the MoE release. That's what I'm running on a Framework Desktop and I get ~50 tok/s plus or minus a few. The dense one is similarly slow for me -- not sure what to expect on your hardware from the MoE but it should probably be much faster.

Post by SeriousM » Wed, May 20, 2026, 6:41 PM UTC

briga wrote: post #119

plufz wrote: post #113

Earlier quoted context omitted.

Which exact model are you using? And with which parameters and quant? And on what hardware? Are you using any specific MCPs or other tools to optimize performance like context-mode or dynamic context pruning? I’ve used local models a reasonable amount before but I’m just starting out with opencode. Haven’t had great results yet but really want this to work for simpler tasks. My opencode newly installed is also having…

I'm running Qwen3.6:27b Q4 KM on a 4090 and similarly fast CPU and I think 32GB of RAM. Make sure the context window is set to be big enough otherwise the conversation will keep compacting. No special MCP tools set up yet. Qwen is able to do web search out-of-the-box although I think it is getting blocked by anti-bot firewalls--I still need to figure out if I can fix that.

Post by czottmann » Wed, May 20, 2026, 6:46 PM UTC

InsideOutSanta wrote: post #39

howmayiannoyyou wrote: post #29

I can't bring myself to use any model that trains or sends telemetry back to my country's primary competitor/adversary. I don't care how much money is saved.

As somebody in Europe, uh, that doesn't leave many options.

Post by MintPaw » Wed, May 20, 2026, 6:58 PM UTC

adjejmxbdjdn wrote: post #41

embedding-shape wrote: post #27

Earlier quoted context omitted.

Unless US hyperscalers do the same in reverse, I hope the status quo stays as it is. Either people are happy to share, and the sharing should happen both ways, or US hyperscalers can keep isolating themselves as they've done so far.

I do hope The U.S. hyperscalers do the same as well. In an ideal world U.S. residents would use Chinese AI models and Chinese residents would use U.S. AI models. Governments in both countries are collecting data for nefarious reasons. But the Chinese government has far less influence on a U.S. resident and vice versa. We are all better off if our data is collected by a government halfway across the world instead of o…