Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-35B-A3B: Agentic coding power, now open to all

qwen.ai

Post reply on HN
Print view View original

121–130 of 563 posts

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#121

Post by wongarsu »

dunb wrote: post #78
zshn25 wrote: post #59

What do all the numbers 6-35B-A3B mean?

3.6 is the release version for Qwen. This model is a mixture of experts (MoE), so while the total model size is big (35 billion parameters), each forward pass only activates a portion of the network that’s most relevant to your request (3 billion active parameters). This makes the model run faster, especially if you don’t have enough VRAM for the whole thing. The performance/intelligence is said to be about the same…

And even if you have enough VRAM to fit the entire thing, inference speed after the first token is proportional to (activated parameters)/(vram bandwidth)

If you have the vram to spare, a model with more total params but fewer activated ones can be a very worthwhile tradeoff. Of course that's a big if

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#122

Post by ekianjo »

terataiijo wrote: post #57
homebrewer wrote: post #18

Already quantized/converted into a sane format by Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

lmao they are so fast yooo

yeah and often their quants are broken. They had to update their Gemma4 quants like 4 times in the past 2 weeks.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#123

Post by zkmon »

Miraste wrote: post #110
zozbot234 wrote: post #62

Earlier quoted context omitted.

The 27B model is dense. Releasing a dense model first would be terrible marketing, whereas 35A3B is a lot smarter and more quick-witted by comparison!

What? 35B-A3B is not nearly as smart as 27B.

Yes.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#124

Post by ekianjo »

Miraste wrote: post #110
zozbot234 wrote: post #62

Earlier quoted context omitted.

The 27B model is dense. Releasing a dense model first would be terrible marketing, whereas 35A3B is a lot smarter and more quick-witted by comparison!

What? 35B-A3B is not nearly as smart as 27B.

yeah the 27B feels like something completely different. If you use it on long context tasks it performs WAY better than 35b-a3b

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#125

Post by cjbgkagh »

utilize1808 wrote: post #111
palmotea wrote: post #97

Earlier quoted context omitted.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

Obviously going to depend on your definition of "decent". My impression so far is that you will need between 90GB to 100GB of memory to run medium sized (31B dense or ~110B MoE) models with some quantization enabled.

I’m running Gemma4 31B (Q8) on my 2 4090s (48GB) with no problem.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#126

Post by tristor »

adrian_b wrote: post #120
tristor wrote: post #85

I'm disappointed they didn't release a 27B dense model. I've been working with Qwen3.5-27B and Qwen3.5-35B-A3B locally, both in their native weights and the versions the community distilled from Opus 4.6 (Qwopus), and I have found I generally get higher quality outputs from the 27B dense model than the 35B-A3B MOE model. My basic conclusion was that MoE approach may be more memory efficient, but it requires a fairly…

You are right, but this is just the first open-weights model of this family. They said that they will release several open-weights models, though there was an implication that they might not release the biggest models.

I'm totally fine with that, frankly. I'm blessed with 128GB of Unified Memory to run local models, but that's still tiny in comparison the larger frontier models. I'd much rather get a full array of small and medium sized models, and building useful things within the limits of smaller models is more interesting to me anyway.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#127

Post by angoragoats »

palmotea wrote: post #97
trvz wrote: post #84

Earlier quoted context omitted.

If you have to ask then your GPU is too small. With 16 GB you'll be only able to run a very compressed variant with noticable quality loss.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

Macs with unified memory are economical in terms of $/GB of video memory, and they match an optimized/home built GPU setup in efficiency (W/token), but they are slow in terms of absolute performance.

With this model, since the number of active parameters is low, I would think that you would be fine running it on your 16GB card, as long as you have, say 32GB of regular system memory. Temper your expectations about speed with this setup, as your system memory and CPU are multiple times slower than the GPU, so when layers spill over you will slow down.

To avoid this, there's no need to buy a Mac -- a second 16GB GPU would do the trick just fine, and the combined dual GPU setup will likely be faster than a cheap mac like a Mac mini. Pay attention to your PCIe slots, but as long as you have at least an x4 slot for the second GPU, you'll be fine (LLM inference doesn't need x8 or x16).

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#128

Post by adrian_b »

bossyTeacher wrote: post #39
Havoc wrote: post #23

Earlier quoted context omitted.

The Chinese models are generally pretty good. > Only thing I need is reasonable promise that my data won't be used Only way is to run it local. I personally don’t worry about this too much. Things like medical questions I tend to do against local models though

Have you tried asking about sensitive topics? I asked it if there were out of bounds topics but it never gave me a list. See its responses: Convo 1 - Q: ok tell me about taiwan - A: Oops! There was an issue connecting to Qwen3.6-Plus. Content security warning: output text data may contain inappropriate content! Convo 2 - Q: is winnie the pooh broadcasted in china? - A: Oops! There was an issue connecting to Qwen3.6-P…

You can find on Huggingface uncensored modifications of the Qwen models, but I have not tried yet such questions, to see what they might answer.

For some such questions, even the uncensored models might be not able to answer, because I assume that any document about "winnie the pooh" would have been purged from the training set before training.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#129

Post by zackangelo »

stingraycharles wrote: post #69
zozbot234 wrote: post #40

Earlier quoted context omitted.

This is just one model in the Qwen 3.6 series. They will most likely release the other small sizes (not much sense in keeping them proprietary) and perhaps their 122A10B size also, but the flagship 397A17B size seems to have been excluded.

397A17B = 397B total weights, 17B per expert?

17b per token. So when you’re generating a single stream of text (“decoding”) 17b parameters are active.

If you’re decoding multiple streams, it will be 17b per stream (some tokens will use the same expert, so there is some overlap).

When the model is ingesting the prompt (“prefilling”) it’s looking at many tokens at once, so the number of active parameters will be larger.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#130

Post by layer8 »

palmotea wrote: post #97
trvz wrote: post #84

Earlier quoted context omitted.

If you have to ask then your GPU is too small. With 16 GB you'll be only able to run a very compressed variant with noticable quality loss.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

It’s also doable with AMD Strix Halo.
Post reply on HN