Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-35B-A3B: Agentic coding power, now open to all

qwen.ai

Post reply on HN
Print view View original

171–180 of 563 posts

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#171

Post by danielhanchen »

bildung wrote: post #168
danielhanchen wrote: post #161

Earlier quoted context omitted.

1. Gemma-4 we re-uploaded 4 times - 3 times were 10-20 llama.cpp bug fixes - we had to notify people to upload the correct ones. The 4th is an official Gemma chat template improvement from Google themselves. 2. Qwen3.5 - we shared our 7TB research artifacts showing which layers not to quantize - all provider's quants were under optimized, not broken - ssm_out and ssm_* tensors were the issue - we're now the best in t…

Fair enough, appreciate the detailed response! Can you elaborate why other quantizations weren't affected (e.g. bartowski)? Simply because they were straight Q4 etc. for every layer?

No Bartowski's are more affected - (38% NaN) than ours (22%) - for MiniMax 2.7 see https://www.reddit.com/r/LocalLLaMA/comments/1slk4di/minimax...

We already fixed ours. Bart hasn't yet but is still working on it following our findings.

blk.61.ffn_down_exps in Q4_K or Q5_K failed - it must be in Q6_K otherwise it overflows.

For the others, yes layers in some precision don't work. For eg Qwen3.5 ssm_out must be minimum Q4-Q6_K.

ssm_alpha and ssm_beta must be Q8_0 or higher.

Again Bart and others apply our findings - see https://www.reddit.com/r/LocalLLaMA/comments/1rgel19/new_qwe...

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#172

Post by Der_Einzige »

ekianjo wrote: post #124
Miraste wrote: post #110

Earlier quoted context omitted.

What? 35B-A3B is not nearly as smart as 27B.

yeah the 27B feels like something completely different. If you use it on long context tasks it performs WAY better than 35b-a3b

I've been telling analysts/investors for a long time that dense architectures aren't "worse" than sparse MoEs and to continue to anticipate the see-saw of releases on those two sub-architectures. Glad to continuously be vindicated on this one.

For those who don't believe me. Go take a look at the logprobs of a MoE model and a dense model and let me know if you can notice anything. Researchers sure did.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#173

Post by Aurornis »

WithinReason wrote: post #106
palmotea wrote: post #67

Earlier quoted context omitted.

How much VRAM does it need? I haven't run a local model yet, but I did recently pick up a 16GB GPU, before they were discontinued.

It's on the page: Precision Quantization Tag File Size 1-bit UD-IQ1_M 10 GB 2-bit UD-IQ2_XXS 10.8 GB UD-Q2_K_XL 12.3 GB 3-bit UD-IQ3_XXS 13.2 GB UD-Q3_K_XL 16.8 GB 4-bit UD-IQ4_XS 17.7 GB UD-Q4_K_XL 22.4 GB 5-bit UD-Q5_K_XL 26.6 GB 16-bit BF16 69.4 GB

Additional VRAM is needed for context.

This model is a MoE model with only 3B active parameters per expert which works well with partial CPU offload. So in practice you can run the -A(N)B models on systems that have a little less VRAM than you need. The more you offload to the CPU the slower it becomes though.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#174

Post by kamranjon »

aliljet wrote: post #73

I'm broadly curious how people are using these local models. Literally, how are they attaching harnesses to this and finding more value than just renting tokens from Anthropic of OpenAI?

I use LMStudio to host and run GLM 4.7 Flash as a coding agent. I use it with the Pi coding agent, but also use it with the Zed editor agent integrations. I've used the Qwen models in the past, but have consistently come back to GLM 4.7 because of its capabilities. I often use Qwen or Gemma models for their vision capabilities. For example, I often will finish ML training runs, take a photo of the graphs and visualizations of the run metrics and ask the model to tell me things I might look at tweaking to improve subsequent training runs. Qwen 3.5 0.8b is pretty awesome for really small and quick vision tasks like "Give me a JSON representation of the cards on this page".

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#175

Post by Ladioss »

txtsd wrote: post #54
homebrewer wrote: post #18

Already quantized/converted into a sane format by Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

So I can use this in claude code with `ollama run claude`?

More like `ollama launch claude --model qwen3.6:latest`

Also you need to check your context size, Ollama default to 4K if <24 Gb of VRAM and you need 64K minimum if you want claude to be able to at least lift a finger.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#176

Post by Aurornis »

homebrewer wrote: post #18

Already quantized/converted into a sane format by Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

Unsloth is great for uploading quants quickly to experiment with, but everyone should know that they almost always revise their quants after testing.

If you download the release day quants with a tool that doesn’t automatically check HF for new versions you should check back again in a week to look for updated versions.

Some times the launch day quantizations have major problems which leads to early adopters dismissing useful models. You have to wait for everyone to test and fix bugs before giving a model a real evaluation.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#177

Post by aesthesia »

adrian_b wrote: post #80
julianlam wrote: post #21

Earlier quoted context omitted.

Can you share more about what adaptations you made when using smaller models? I'm just starting my exploration of these small models for coding on my 16GB machine (yeah, puny...) and am running into issues where the solution may very well be to reduce the scope of the problem set so the smaller model can handle it.

It is very unlikely that general claims about a model are useful, but only very specific claims, which indicate the exact number of parameters and quantization methods that are used by the compared models. If you perform the inference locally, there is a huge space of compromise between the inference speed and the quality of the results. Most open weights models are available in a variety of sizes. Thus you can choos…

> Anthropic has also admitted that the bugs found by Mythos had not been found by using a prompt like "find the bugs", but by running many times Mythos on each file with increasingly more specific prompts, until the final run that requested only a confirmation of the bug, not searching for it.

Unless there's been more information since their original post (https://red.anthropic.com/2026/mythos-preview/), this is a misleading description of the scaffold. The process was:

- provide a container with running software and its source code

- prompt Mythos to prioritize source files based on the likelihood they contain vulnerabilities

- use this prioritization to prompt parallel agents to look for and verify vulnerabilities, focusing on but not limited to a single seed file

- as a final validation step, have another instance evaluate the validity and interestingness of the resulting bug reports

This amounts to at most three invocations of the model for each file, once for prioritization, once for the main vulnerability run, and once for the final check. The prompts only became more specific as a result of information the model itself produced, not any external process injecting additional information.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#178

Post by bigyabai »

canpan wrote: post #71
kombine wrote: post #46

What kind of hardware (preferably non-Apple) can run this model? What about 122B?

Any good gaming pc can run the 35b-a3 model. Llama cpp with ram offloading. A high end gaming PC can run it at higher speeds. For your 122b, you need a lot of memory, which is expensive now. And it will be much slower as you need to use mostly system ram.

Seconding this. You can get A3B/A4B models to run with 10+ tok/sec on a modern 6/8GB GPU with 32k context if you optimize things well. The cheapest way to run this model at larger contexts is probably a 12gb RTX 3060.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#179

Post by Ladioss »

palmotea wrote: post #67
homebrewer wrote: post #18

Already quantized/converted into a sane format by Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

How much VRAM does it need? I haven't run a local model yet, but I did recently pick up a 16GB GPU, before they were discontinued.

You can run 25-30b model easily if you use Q3 or Q4 quants and llama-server with a pretty long list of options.
Post reply on HN