Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.6-35B-A3B: Agentic coding power, now open to all

qwen.ai

Post reply on HN
Print view View original

401–410 of 563 posts

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#401

Post by mncharity »

rubiquity wrote: post #364
mncharity wrote: post #347

Earlier quoted context omitted.

Fwiw, with its predecessor's Qwen3.5-35B-A3B-Q6_K.gguf, on a laptop's 6 GB VRAM and 32 GB RAM, with default llama.cpp settings, I get 20 t/s generation.

Have you tried running llama.cpp with Unified Memory Access[1] so your iGPU can seamlessly grab some of the RAM? The environment variable is prefixed with CUDA but this is not CUDA specific. It made a pretty significant difference (> 40% tg/s) on my Ryzen 7840U laptop. 1 - https://github.com/ggml-org/llama.cpp/blob/master/docs/build...

I hadn't tried that, thanks! I found simply defining GGML_CUDA_ENABLE_UNIFIED_MEMORY, whether 1, 0, or "", was a 10x hit to 2 t/s. Perhaps because the laptop's RAM is already so over-committed there. But with the much smaller 4B Qwen3.5-4B-Q8_0.gguf, it doubled performance from 20 to 40+ t/s! Tnx! (an old Quadro RTX 3000 rather than an iGPU)

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#402

Post by adrian_b »

boppo1 wrote: post #322
coder543 wrote: post #165

Earlier quoted context omitted.

Not true. With a MoE, you can offload quite a bit of the model to CPU without losing a ton of performance. 16GB should be fine to run the 4-bit (or larger) model at speeds that are decent. The --n-cpu-moe parameter is the key one on llama-server, if you're not just using -fit on.

I've been way out of the local game for a while now, what's the best way to run models for a fairly technical user? I was using llama.cpp in the command line before and using bash files for prompts.

Running llama-server (it belongs to llama.cpp) starts a HTTP server on a specified port.

You can connect to that port with any browser, for chat.

Or you can connect to that port with any application that supports the OpenAI API, e.g. a coding assistant harness.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#403

Post by kylehotchkiss »

kridsdale3 wrote: post #272
kylehotchkiss wrote: post #252

Earlier quoted context omitted.

How many people/hackernews can run a 397b param model at home? Probably like 20-30.

I can (barely, but sustainably) run Q3.5 397B on my Mac Studio with 256GB unified. It cost $10,000 but that's well within reach for most people who are here, I expect.

you have proved my point

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#404

Post by chabes »

bossyTeacher wrote: post #16

Does anyone have any experience with Qwen or any non-Western LLMs? It's hard to get a feel out there with all the doomerists and grifters shouting. Only thing I need is reasonable promise that my data won't be used for training or at least some of it won't. Being able to export conversations in bulk would be helpful.

Run open models locally. Data stays local, and exporting sessions is straightforward.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#405

Post by GistNoesis »

simonw wrote: post #270

I've been running this on my laptop with the Unsloth 20.9GB GGUF in LM Studio: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF/blob/mai... It drew a better pelican riding a bicycle than Opus 4.7 did! https://simonwillison.net/2026/Apr/16/qwen-beats-opus/

Thanks for pointing to the GGUF.

I just tried this GGUF with llama.cpp in its UD Q4_K_XL version on my custom agentic oritened task consisiting of wiki exploration and automatic database building ( https://github.com/GistNoesis/Shoggoth.db/ )

I noted a nice improvement over QWen3.5 in its ability to discover new creatures in the open ended searching task, but I've not quantified it yet with numbers. It also seems faster, at around 140 token/s compared to 100 token/s , but that's maybe due to some different configuration options.

Some little difference with QWen3.5 : to avoid crashes due to lack of memory in multimodal I had to pass --no-mmproj-offload to disable the gpu offload to convert the images to tokens otherwise it would crash for high resolutions images. I also used quantized kv store by passing -ctk q8_0 -ctv q8_0 and with a ctx-size 150000 it only need 23099 MiB of device memory which means no partial RAM offloading when I use a RTX 4090.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#406

Post by jaspanglia »

simonw wrote: post #270

I've been running this on my laptop with the Unsloth 20.9GB GGUF in LM Studio: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF/blob/mai... It drew a better pelican riding a bicycle than Opus 4.7 did! https://simonwillison.net/2026/Apr/16/qwen-beats-opus/

The real question is what the next truly weird, un-optimized prompt will be. Something involving a sloth debugging a quantum computer in MS Paint?"

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#407

Post by arcanemachiner »

est wrote: post #229
WithinReason wrote: post #106

Earlier quoted context omitted.

It's on the page: Precision Quantization Tag File Size 1-bit UD-IQ1_M 10 GB 2-bit UD-IQ2_XXS 10.8 GB UD-Q2_K_XL 12.3 GB 3-bit UD-IQ3_XXS 13.2 GB UD-Q3_K_XL 16.8 GB 4-bit UD-IQ4_XS 17.7 GB UD-Q4_K_XL 22.4 GB 5-bit UD-Q5_K_XL 26.6 GB 16-bit BF16 69.4 GB

I really want to know what does M, K, XL XS mean in this context and how to choose. I searched all unsloth doc and there seems no explaination at all.

Just start with q4_k_m and figure out the rest later.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#408

Post by storus »

zozbot234 wrote: post #387
storus wrote: post #385

Earlier quoted context omitted.

They no longer show reasoning traces and are throttling more aggressively.

They never showed full reasoning traces, just post-hoc summaries.

DeepSeek still shows them, it sometimes says "I am ChatGPT", and Claude sometimes says "I am DeepSeek" so the distillation went both ways.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#409

Post by mncharity »

rubiquity wrote: post #378
zozbot234 wrote: post #366

Earlier quoted context omitted.

Your link seems to be describing a runtime environment variable, it doesn't need a separate build from source. I'm not sure though (1) why this info is in build.md which should be specific to the building process, rather than some separate documentation; and (2) if this really isn't CUDA-specific, why the canonical GGML variable name isn't GGML_ENABLE_UNIFIED_MEMORY , with the _CUDA_ variant treated as a legacy alias…

You are right that it is an environment variable, and that's how I have it set in my nix config. Thanks for correcting that. Unfortunately llama.cpp is somewhat notorious for having lackluster docs. Most of the CLI tools don't even tell you what they are for.

Hmm. Perhaps there's a niche for a "The Missing Guide to llama.cpp"? Getting started, I did things like wrapping llama-cli in a pty... and only later noticing a --simple-io argument. I wonder if "living documents" are a thing yet, where LLMs keep an eye on repo and fora, and update a doc autonomously.

Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all

#410

Post by jonaustin »

zozbot234 wrote: post #40
bertili wrote: post #3

A relief to see the Qwen team still publishing open weights, after the kneecapping [1] and departures of Junyang Lin and others [2]! [1] https://news.ycombinator.com/item?id=47246746 [2] https://news.ycombinator.com/item?id=47249343

This is just one model in the Qwen 3.6 series. They will most likely release the other small sizes (not much sense in keeping them proprietary) and perhaps their 122A10B size also, but the flagship 397A17B size seems to have been excluded.

And shout-out to Qwen if they release 122b -- Jeff Barr's original Gemma 4 tweet said they'd release a ~122b, then it got redacted :(
Post reply on HN