Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-35B-A3B: Agentic coding power, now open to all

qwen.ai

Post reply on HN
Print view View original

101–110 of 563 posts

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#101

Post by zshn25 »

dunb wrote: post #78
zshn25 wrote: post #59

What do all the numbers 6-35B-A3B mean?

3.6 is the release version for Qwen. This model is a mixture of experts (MoE), so while the total model size is big (35 billion parameters), each forward pass only activates a portion of the network that’s most relevant to your request (3 billion active parameters). This makes the model run faster, especially if you don’t have enough VRAM for the whole thing. The performance/intelligence is said to be about the same…

Sorry, how did you calculate the 10.25B?

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#102

Post by daemonologist »

kombine wrote: post #46

What kind of hardware (preferably non-Apple) can run this model? What about 122B?

The 3B active is small enough that it's decently fast even with experts offloaded to system memory. Any PC with a modern (>=8 GB) GPU and sufficient system memory (at least ~24 GB) will be able to run it okay; I'm pretty happy with just a 7800 XT and DDR4. If you want faster inference you could probably squeeze it into a 24 GB GPU (3090/4090 or 7900 XTX) but 32 GB would be a lot more comfortable (5090 or Radeon Pro).

122B is a more difficult proposition. (Also, keep in mind the 3.6 122B hasn't been released yet and might never be.) With 10B active parameters offloading will be slower - you'd probably want at least 4 channels of DDR5, or 3x 32GB GPUs, or a very expensive Nvidia Pro 6000 Blackwell.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#104

Post by bfivyvysj »

palmotea wrote: post #97
trvz wrote: post #84

Earlier quoted context omitted.

If you have to ask then your GPU is too small. With 16 GB you'll be only able to run a very compressed variant with noticable quality loss.

> If you have to ask then your GPU is too small. What's the minimum memory you need to run a decent model? Is it pretty much only doable by people running Macs with unified memory?

A bit like asking how long is a piece of string.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#105

Post by mildred593 »

kombine wrote: post #46

What kind of hardware (preferably non-Apple) can run this model? What about 122B?

I can run this on an AMD Framework laptop. A Ryzen 7 (I dont have Ryzen AI, just Ryzen 7 7840U) with 32+48 GB DDR. The Ryzen unified memory is enough, I get 26GB of VRAM at least.

Fedora 43 and LM Studio with Vulkan llama.cpp

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#106

Post by WithinReason »

palmotea wrote: post #67
homebrewer wrote: post #18

Already quantized/converted into a sane format by Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

How much VRAM does it need? I haven't run a local model yet, but I did recently pick up a 16GB GPU, before they were discontinued.

It's on the page:

  Precision  Quantization Tag File Size
  1-bit      UD-IQ1_M         10 GB
  2-bit      UD-IQ2_XXS       10.8 GB
             UD-Q2_K_XL       12.3 GB
  3-bit      UD-IQ3_XXS       13.2 GB
             UD-Q3_K_XL       16.8 GB
  4-bit      UD-IQ4_XS        17.7 GB
             UD-Q4_K_XL       22.4 GB
  5-bit      UD-Q5_K_XL       26.6 GB
  16-bit     BF16             69.4 GB

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#107

Post by samtheprogram »

ru552 wrote: post #81
kombine wrote: post #46

What kind of hardware (preferably non-Apple) can run this model? What about 122B?

You won't like it, but the answer is Apple. The reason is the unified memory. The GPU can access all 32gb, 64gb, 128gb, 256gb, etc. of RAM. An easy way (napkin math) to know if you can run a model based on it's parameter size is to consider the parameter size as GB that need to fit in GPU RAM. 35B model needs atleast 35gb of GPU RAM. This is a very simplified way of looking at it and YES, someone is going to say you…

That estimate doesn't account for context, which is very important for tool use and coding.

I used this napkin math for image generation, since the context (prompts) were so small, but I think it's misleading at best for most uses.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#108

Post by zozbot234 »

palmotea wrote: post #67
homebrewer wrote: post #18

Already quantized/converted into a sane format by Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

How much VRAM does it need? I haven't run a local model yet, but I did recently pick up a 16GB GPU, before they were discontinued.

Should run just fine with CPU-MoE and mmap, but inference might be a bit slow if you have little RAM.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#109

Post by terramex »

kombine wrote: post #46

What kind of hardware (preferably non-Apple) can run this model? What about 122B?

I run Gemma 4 26B-A4B with 256k context (maximum) on Radeon 9070XT 16GB VRAM + 64GB RAM with partial GPU offload (with recommended LMStudio settings) at very reasonable 35 tokens per second, this model is similiar in size so I expect similar performance.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#110

Post by Miraste »

zozbot234 wrote: post #62
armanj wrote: post #20

I recall a Qwen exec posted a public poll on Twitter, asking which model from Qwen3.6 you want to see open-sourced; and the 27b variant was by far the most popular choice. Not sure why they ignored it lol.

The 27B model is dense. Releasing a dense model first would be terrible marketing, whereas 35A3B is a lot smarter and more quick-witted by comparison!

What? 35B-A3B is not nearly as smart as 27B.
Post reply on HN