Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-35B-A3B: Agentic coding power, now open to all

qwen.ai

Post reply on HN
Print view View original

111–120 of 563 posts

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#111

Post by utilize1808 »

palmotea wrote: post #97
trvz wrote: post #84

Earlier quoted context omitted.

If you have to ask then your GPU is too small. With 16 GB you'll be only able to run a very compressed variant with noticable quality loss.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

Obviously going to depend on your definition of "decent". My impression so far is that you will need between 90GB to 100GB of memory to run medium sized (31B dense or ~110B MoE) models with some quantization enabled.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#112

Post by littlestymaar »

palmotea wrote: post #97
trvz wrote: post #84

Earlier quoted context omitted.

If you have to ask then your GPU is too small. With 16 GB you'll be only able to run a very compressed variant with noticable quality loss.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

No, GP is excessively restrictive. Llama.cpp supports RAM offloading out of the box.

It's going to be slower than if you put everything on your GPU but it would work.

And if it's too slow for your taste you can try the quantized version (some Q3 variant should fit) and see how well it works for you.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#113

Post by zkmon »

armanj wrote: post #20

I recall a Qwen exec posted a public poll on Twitter, asking which model from Qwen3.6 you want to see open-sourced; and the 27b variant was by far the most popular choice. Not sure why they ignored it lol.

I'm guessing 3.5-27b would beat 3.6-35b. MoE is a bad idea. Because for the same VRAM 27b would leave a lot more room, and the quality of work directly depends on context size, not just the "B" number.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#114

Post by arxell »

zozbot234 wrote: post #62
armanj wrote: post #20

I recall a Qwen exec posted a public poll on Twitter, asking which model from Qwen3.6 you want to see open-sourced; and the 27b variant was by far the most popular choice. Not sure why they ignored it lol.

The 27B model is dense. Releasing a dense model first would be terrible marketing, whereas 35A3B is a lot smarter and more quick-witted by comparison!

Each has it's pros and cons. Dense models of equivalent total size obviously do run slower if all else is equal, however, the fact is that 35A3B is absolutely not 'a lot smarter'... in fact, if you set aside the slower inference rates, Qwen3.5 27B is arguably more intelligent and reliable. I use both regularly on a Strix Halo system... the Just see the comparison table here: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF . The problem that you have to acknowledge if running locally (especially for coding tasks) is that your primary bottleneck quickly becomes prompt processing (NOT token generation) and here the differences between dense and MOE are variable and usually negligible.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#115

Post by palmotea »

bfivyvysj wrote: post #104
palmotea wrote: post #97

Earlier quoted context omitted.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

A bit like asking how long is a piece of string.

More like "about how long of a string do I need to run between two houses in the densest residential neighborhood of single-family homes in the US?"

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#116

Post by Havoc »

bossyTeacher wrote: post #39
Havoc wrote: post #23

Earlier quoted context omitted.

The Chinese models are generally pretty good. > Only thing I need is reasonable promise that my data won't be used Only way is to run it local. I personally don’t worry about this too much. Things like medical questions I tend to do against local models though

Have you tried asking about sensitive topics? I asked it if there were out of bounds topics but it never gave me a list. See its responses: Convo 1 - Q: ok tell me about taiwan - A: Oops! There was an issue connecting to Qwen3.6-Plus. Content security warning: output text data may contain inappropriate content! Convo 2 - Q: is winnie the pooh broadcasted in china? - A: Oops! There was an issue connecting to Qwen3.6-P…

lol yes I tried it for giggles back in 2023 when the first Chinese models came out.

Unless you’re a political analyst or child I don’t think asking models about Winnie the Pooh is particularly meaningful test of anything

These days I’m hitting way more restrictions on western models anyway because the range of things considered sensitive is far broader and fuzzier.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#117

Post by marssaxman »

aliljet wrote: post #73

I'm broadly curious how people are using these local models. Literally, how are they attaching harnesses to this and finding more value than just renting tokens from Anthropic of OpenAI?

I used vLLM and qwen3-coder-next to batch-process a couple million documents recently. No token quota, no rate limits, just 100% GPU utilization until the job was done.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#118

Post by Panda4 »

aliljet wrote: post #73

I'm broadly curious how people are using these local models. Literally, how are they attaching harnesses to this and finding more value than just renting tokens from Anthropic of OpenAI?

I was thinking the same thing. My only guess is that they are excited about local models because they can run it cheaper through Open Router ?

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#119

Post by postalcoder »

pj_mukh wrote: post #89
txtsd wrote: post #54

Earlier quoted context omitted.

So I can use this in claude code with `ollama run claude`?

have you found a model that does this with usable speeds on an M2/M3?

On a M4 MBP ollama's qwen3.5:35b-a3b-coding-nvfp4 runs incredibly fast when in the claude/codex harness. M2/M3 should be similar.

It's incomparably faster than any other model (i.e. it's actually usable without cope). Caching makes a huge difference.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#120

Post by adrian_b »

tristor wrote: post #85

I'm disappointed they didn't release a 27B dense model. I've been working with Qwen3.5-27B and Qwen3.5-35B-A3B locally, both in their native weights and the versions the community distilled from Opus 4.6 (Qwopus), and I have found I generally get higher quality outputs from the 27B dense model than the 35B-A3B MOE model. My basic conclusion was that MoE approach may be more memory efficient, but it requires a fairly…

You are right, but this is just the first open-weights model of this family.

They said that they will release several open-weights models, though there was an implication that they might not release the biggest models.

Post reply on HN