Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-35B-A3B: Agentic coding power, now open to all

qwen.ai

Post reply on HN
Print view View original

161–170 of 563 posts

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#161

Post by danielhanchen »

bildung wrote: post #142
terataiijo wrote: post #57

Earlier quoted context omitted.

lmao they are so fast yooo

Bad QA :/ They had a bunch of broken quantizations in the last releases

1. Gemma-4 we re-uploaded 4 times - 3 times were 10-20 llama.cpp bug fixes - we had to notify people to upload the correct ones. The 4th is an official Gemma chat template improvement from Google themselves.

2. Qwen3.5 - we shared our 7TB research artifacts showing which layers not to quantize - all provider's quants were under optimized, not broken - ssm_out and ssm_* tensors were the issue - we're now the best in terms of KLD and disk space

3. MiniMax 2.7 - we swiftly fixed it due to NaN PPL - we found the issue in all quants regardless of provider - so it affected everyone not just us. We wrote a post on it, and fixed it - others have taken our fix and fixed their quants, whilst some haven't updated.

Note we also fixed bugs in many OSS models like Gemma 1, Gemma 3, Llama chat template fixes, Mistral, and many more.

Unfortunately sometimes quants break, but we fix them quickly, and 95% of times these are out of our hand.

We swiftly and quickly fix them, and write up blogs on what happened. Other providers simply just take our blogs and fixes and re-apply, re-use our fixes.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#162

Post by sosodev »

zackangelo wrote: post #138
sosodev wrote: post #86

Earlier quoted context omitted.

These are not autocomplete models. It’s built to be used with an agentic coding harness like Pi or OpenCode.

They are but the IDE needs to be integrated with them. Qwen specifically calls out FIM (“fill in the middle”) support on the model card and you can see it getting confused and posting the control tokens in the example here.

Oh, that’s interesting. Thanks for the correction. I didn’t know such heavily post trained models could still do good ol fashion autocomplete.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#163

Post by zozbot234 »

abhikul0 wrote: post #155
zozbot234 wrote: post #64

Earlier quoted context omitted.

CPU-MoE still helps with mmap. Should not overly hurt token-gen speed on the Mac since the CPU has access to most (though not all) of the unified memory bandwidth, which is the bottleneck.

I'll try to use that, but llama-server has mmap on by default and the model still takes up the size of the model in RAM, not sure what's going on.

Try running CPU-only inference to troubleshoot that. GPU layers will likely just ignore mmap.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#164

Post by bildung »

kombine wrote: post #46

What kind of hardware (preferably non-Apple) can run this model? What about 122B?

I currently run the qwen3.5-122B (Q4) on a Strix Halo (Bosgame M5) and am pretty happy with it. Obviously much slower than hosted models. I get ~ 20t/s with empty context and am down to about 14t/s with 100k of context filled.

No tuning at all, just apt install rocm and rebuilding llama.cpp every week or so.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#165

Post by coder543 »

trvz wrote: post #84
palmotea wrote: post #67

Earlier quoted context omitted.

How much VRAM does it need? I haven't run a local model yet, but I did recently pick up a 16GB GPU, before they were discontinued.

If you have to ask then your GPU is too small. With 16 GB you'll be only able to run a very compressed variant with noticable quality loss.

Not true. With a MoE, you can offload quite a bit of the model to CPU without losing a ton of performance. 16GB should be fine to run the 4-bit (or larger) model at speeds that are decent. The --n-cpu-moe parameter is the key one on llama-server, if you're not just using -fit on.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#166

Post by nurettin »

I tried the car wash puzzle:

You want to wash your car. Car wash is 50m away. Should you walk or go by car?

> Walk. At 50 meters, the round trip is roughly 100 meters, taking about two minutes on foot. Driving would require starting the engine, navigating, parking, and dealing with unnecessary wear for a negligible distance. Walk to the car wash, and if the bay requires the vehicle inside, have it moved there or return on foot. Walking is faster and more efficient.

Classic response. It was really hard to one shot this with Qwen3.5 Q4_K_M.

Qwen3.6 UD-IQ4_XS also failed the first time, then I added this to the system prompt:

> Double check your logic for errors

Then I created a new dialog and asked the puzzle and it responded:

> Drive it. The car needs to be present to be washed. 50 meters is roughly a 1-minute walk or a 10-second drive. Walking leaves the car behind, making the wash impossible. Driving it the short distance is the only option that achieves the goal.

Now 3.6 gets it right every time. So not as great as a super model, but definitely an improvement.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#167

Post by Havoc »

bossyTeacher wrote: post #156
Havoc wrote: post #116

Earlier quoted context omitted.

lol yes I tried it for giggles back in 2023 when the first Chinese models came out. Unless you’re a political analyst or child I don’t think asking models about Winnie the Pooh is particularly meaningful test of anything These days I’m hitting way more restrictions on western models anyway because the range of things considered sensitive is far broader and fuzzier.

> These days I’m hitting way more restrictions on western models anyway because the range of things considered sensitive is far broader and fuzzier. Ah interesting, what are some topics where you are not getting answers?

General chatbot use about daily life. Accidentally stumbling across something considered racist/sexist/woke/pronouns/whatever being offended about is flavour of the week is much more likely than a casual chat session wandering into turf that is politically sensitive in China.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#168

Post by bildung »

danielhanchen wrote: post #161
bildung wrote: post #142

Earlier quoted context omitted.

Bad QA :/ They had a bunch of broken quantizations in the last releases

1. Gemma-4 we re-uploaded 4 times - 3 times were 10-20 llama.cpp bug fixes - we had to notify people to upload the correct ones. The 4th is an official Gemma chat template improvement from Google themselves. 2. Qwen3.5 - we shared our 7TB research artifacts showing which layers not to quantize - all provider's quants were under optimized, not broken - ssm_out and ssm_* tensors were the issue - we're now the best in t…

Fair enough, appreciate the detailed response! Can you elaborate why other quantizations weren't affected (e.g. bartowski)? Simply because they were straight Q4 etc. for every layer?

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#169

Post by recov »

dataflow wrote: post #49

I'm a newbie here and lost how I'm supposed to use these models for coding. When I use them with Continue in VSCode and start typing basic C: #include int m I get nonsensical autocompletions like: #include int m What is going on?

I would use something like zeta-2 instead - https://huggingface.co/bartowski/zed-industries_zeta-2-GGUF

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#170

Post by lopsotronic »

Dangit, I'll need to give this a run on my personal machine. This looks impressive.

At the time of writing, all deepseek or qwen models are de facto prohibited in govcon, including local machine deployments via Ollama or similar. Although no legislative or executive mandate yet exists [1], it's perceived as a gap [2], and contracts are already including language for prohibition not just in the product but any part of the software environment.

The attack surface for a (non-agentic) model running in local ollama is basically non-existent . . but, eh . . I do get it, at some level. While they're not l33t haXX0ring your base, the models are still largely black boxes, can move your attention away from things, or towards things, with no one being the wiser. "Landing Craft? I see no landing craft". This would boil out in test, ideally, but hey, now you know how much time your typical defense subcon spends in meaningful software testing[3].

[1] See also OMB Memorandum M-25-22 (preference for AI developed and produced in the United States), NIST CAISI assessment of PRC-origin AI models as "adversary AI" (September 2025), and House Select Committee on the CCP Report (April 16, 2025), "DeepSeek Unmasked".

[2] Overall, rather than blacklist, I'd recommend a "whitelist" of permitted models, maintained dynamically. This would operate the same way you would manage libraries via SSCG/SSCM (software supply chain governance/management) . . but few if any defense subcons have enough onboard savvy to manage SSCG let alone spooling a parallel construct for models :(. Soooo . . ollama regex scrubbing it is.

[3] i.e. none at all, we barely have the ability to MAKE anything like software, given the combination of underwhelming pay scales and the fact defense companies always seem to have a requirement for on-site 100% in some random crappy town in the middle of BFE. If it wasn't for the downturn in tech we wouldn't have anyone useful at all, but we snagged some silcon refugees.

Post reply on HN