Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-35B-A3B: Agentic coding power, now open to all

qwen.ai

Post reply on HN
Print view View original

141–150 of 563 posts

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#141

Post by [deleted] »

rhdunn wrote: post #72
kombine wrote: post #46

What kind of hardware (preferably non-Apple) can run this model? What about 122B?

The Q5 quantization (26.6GB) should easily run on a 32GB 5090. The Q4 (22.4GB) should fit on a 24GB 4090, but you may need to drop it down to Q3 (16.8GB) when factoring in the context. You can also run those on smaller cards by configuring the number of layers on the GPU. That should allow you to run the Q4/Q5 version on a 4090, or on older cards. You could also run it entirely on the CPU/in RAM if you have 32GB (or…

[deleted]

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#142

Post by bildung »

terataiijo wrote: post #57
homebrewer wrote: post #18

Already quantized/converted into a sane format by Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

lmao they are so fast yooo

Bad QA :/ They had a bunch of broken quantizations in the last releases

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#143

Post by latentsea »

bfivyvysj wrote: post #104
palmotea wrote: post #97

Earlier quoted context omitted.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

A bit like asking how long is a piece of string.

It's twice as long as from one end to the middle.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#144

Post by giobox »

palmotea wrote: post #97
trvz wrote: post #84

Earlier quoted context omitted.

If you have to ask then your GPU is too small. With 16 GB you'll be only able to run a very compressed variant with noticable quality loss.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

It's worth noting now there are other machines than just Apple that combine a powerful SoC with a large pool of unified memory for local AI use:

> https://www.dell.com/en-us/shop/cty/pdp/spd/dell-pro-max-fcm...

> https://marketplace.nvidia.com/en-us/enterprise/personal-ai-...

> https://frame.work/products/desktop-diy-amd-aimax300/configu...

etc.

But yes, a modern SoC-style system with large unified memory pool is still one of the best ways to do it.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#145

Post by littlestymaar »

stingraycharles wrote: post #69
zozbot234 wrote: post #40

Earlier quoted context omitted.

This is just one model in the Qwen 3.6 series. They will most likely release the other small sizes (not much sense in keeping them proprietary) and perhaps their 122A10B size also, but the flagship 397A17B size seems to have been excluded.

397A17B = 397B total weights, 17B per expert?

That's not how it works. Many people get confused by the “expert” naming, when in reality the key part of the original name “sparse mixture of experts” is sparse.

Experts are just chunks of each layers MLP that are only partially activated by each token, there are thousands of “experts” in such a model (for Qwen3-30BA3, it was 48 layers x 128 “experts” per layer with only 8 active at each token)

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#146

Post by wild_egg »

jake-coworker wrote: post #42

This is surprisingly close to Haiku quality, but open - and Haiku is quite a capable model (many of the Claude Code subagents use it).

Where did you see a haiku comparison? Haiku 4.5 was my daily driver for a month or so before Opus 4.5 dropped and would be unreasonably happy if a local model can give me similar capability

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#147

Post by TechSquidTV »

palmotea wrote: post #97
trvz wrote: post #84

Earlier quoted context omitted.

If you have to ask then your GPU is too small. With 16 GB you'll be only able to run a very compressed variant with noticable quality loss.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

My Mac Studio with 96GB of RAM is maybe just at the low end of passable. It's actually extremely good for local image generation. I could somewhat replace something like Nano Banana comfortably on my machine.

But I don't need Nano Banana very much, I need code. While it can, there's no way I would ever opt to use a local model on my machine for code. It makes so much more sense to spend $100 on Codex, it's genuinely not worth discussing.

For non-thinking tasks, it would be a bit slower, but a viable alternative for sure.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#148

Post by alberto-m »

bossyTeacher wrote: post #16

Does anyone have any experience with Qwen or any non-Western LLMs? It's hard to get a feel out there with all the doomerists and grifters shouting. Only thing I need is reasonable promise that my data won't be used for training or at least some of it won't. Being able to export conversations in bulk would be helpful.

I used Qwen CLI's undescribed “coder_agent” (I guess Qwen 3.5 with size auto-selection) and it was powerful enough to complete 95% of a small hobby project involving coding, reverse engineering and debugging. Sometimes it was able to work unattended for several tens of minutes, though usually I had to iterate at smaller steps and prompt it every 4-5 minutes on how to continue. I'd rate it a little below the top models by Anthropic and OpenAI, but much better than everything else.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#149

Post by zozbot234 »

jchw wrote: post #139
palmotea wrote: post #97

Earlier quoted context omitted.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

32 GiB of VRAM is possible to acquire for less than $1000 if you go for the Arc Pro B70. I have two of them. The tokens/sec is nowhere near AMD or NVIDIA high end, but its unexpectedly kind of decent to use. (I probably need to figure out vLLM though as it doesn't seem like llama.cpp is able to do them justice even seemingly with split mode = row. But still, 30t/s on Gemma 4 (on 26B MoE, not dense) is pretty usable,…

New versions of llama.cpp have experimental split-tensor parallelism, but it really only helps with slow compute and a very fast interconnect, which doesn't describe many consumer-grade systems. For most users, pipeline parallelism will be their best bet for making use of multi-GPU setups.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#150

Post by darrenf »

zshn25 wrote: post #101
dunb wrote: post #78

Earlier quoted context omitted.

3.6 is the release version for Qwen. This model is a mixture of experts (MoE), so while the total model size is big (35 billion parameters), each forward pass only activates a portion of the network that’s most relevant to your request (3 billion active parameters). This makes the model run faster, especially if you don’t have enough VRAM for the whole thing. The performance/intelligence is said to be about the same…

Sorry, how did you calculate the 10.25B?

> > The performance/intelligence is said to be about the same as the geometric mean of the total and active parameter counts. So, this model should be equivalent to a dense model with about 10.25 billion parameters.

> Sorry, how did you calculate the 10.25B?

The geometric mean of two numbers is the square root of their product. Square root of 105 (35*3) is ~10.25.

Post reply on HN