Earlier quoted context omitted.
They obviously collaborate with some of the labs prior to the official release date.
That... is a more plausible explanation I didn't think of.
Yes we collab with them!
Qwen3.6-35B-A3B: Agentic coding power, now open to all
151–160 of 563 posts
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#152I'm broadly curious how people are using these local models. Literally, how are they attaching harnesses to this and finding more value than just renting tokens from Anthropic of OpenAI?
The privacy/data security angle really is important in some regions and industries. Think European privacy laws or customers demanding NDAs. The value of Anthropic and OpenAI is zero for both cases, so easy to beat, despite local models being dumber and slower.
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#153Earlier quoted context omitted.
lmao they are so fast yooo
yeah and often their quants are broken. They had to update their Gemma4 quants like 4 times in the past 2 weeks.
No it's not our fault - re our 4 uploads - the first 3 are due to llama.cpp fixing bugs - this was out of our control (we're llama.cpp contributors, but not the main devs) - we could have waited, but it's best to update when multiple (10-20) bugs are fixed.
The 4th is Google themselves improving the chat template for tool calling for Gemma.
https://github.com/ggml-org/llama.cpp/issues/21255 was another issue CUDA 13.2 was broken - this was NVIDIA's CUDA compiler itself breaking - fully out of our hands - but we provided a solution for it.
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#154I'm disappointed they didn't release a 27B dense model. I've been working with Qwen3.5-27B and Qwen3.5-35B-A3B locally, both in their native weights and the versions the community distilled from Opus 4.6 (Qwopus), and I have found I generally get higher quality outputs from the 27B dense model than the 35B-A3B MOE model. My basic conclusion was that MoE approach may be more memory efficient, but it requires a fairly…
You are right, but this is just the first open-weights model of this family. They said that they will release several open-weights models, though there was an implication that they might not release the biggest models.
Given that DeepSeek, GLM, Kimi etc have all released large open weight models, I am personally grateful that Qwen fills the mid/small sized model gap even if they keep their largest models to themselves. The only other major player in the mid/small sized space at this point is pretty much only Gemma.
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#155Earlier quoted context omitted.
Mac has unified memory, so 36GB is 36GB for everything- gpu,cpu.
CPU-MoE still helps with mmap. Should not overly hurt token-gen speed on the Mac since the CPU has access to most (though not all) of the unified memory bandwidth, which is the bottleneck.
I'll try to use that, but llama-server has mmap on by default and the model still takes up the size of the model in RAM, not sure what's going on.
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#156Earlier quoted context omitted.
Have you tried asking about sensitive topics? I asked it if there were out of bounds topics but it never gave me a list. See its responses: Convo 1 - Q: ok tell me about taiwan - A: Oops! There was an issue connecting to Qwen3.6-Plus. Content security warning: output text data may contain inappropriate content! Convo 2 - Q: is winnie the pooh broadcasted in china? - A: Oops! There was an issue connecting to Qwen3.6-P…
lol yes I tried it for giggles back in 2023 when the first Chinese models came out. Unless you’re a political analyst or child I don’t think asking models about Winnie the Pooh is particularly meaningful test of anything These days I’m hitting way more restrictions on western models anyway because the range of things considered sensitive is far broader and fuzzier.
> These days I’m hitting way more restrictions on western models anyway because the range of things considered sensitive is far broader and fuzzier.
Ah interesting, what are some topics where you are not getting answers?
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#157how does this compare to gpt-oss-120b? It seems weird to leave it out.
GPT-OSS 120B (really 117B-A5.1B) is a lot bigger. better comparison would be to 20B (21B-A3.6B).
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#158I'm broadly curious how people are using these local models. Literally, how are they attaching harnesses to this and finding more value than just renting tokens from Anthropic of OpenAI?
Idk about everyone else, but I don’t want to rent tokens forever. I want a self hosted model that is completely private and can’t be monitored or adulterated without me knowing. I use both currently, but I am excited at the prospect of maybe not having to in the near to mid future.
I’ve increasingly started self hosting everything in my home lately because I got tired of SAAS rug pulls and I don’t see why LLM’s should eventually be any different.
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#159I'm broadly curious how people are using these local models. Literally, how are they attaching harnesses to this and finding more value than just renting tokens from Anthropic of OpenAI?
Qwen3.5-9B has been extremely useful for local fuzzy table extraction OCR for data that cannot be sent to the cloud.
The documents have subtly different formatting and layout due to source variance. Previously we used a large set of hierarchical heuristics to catch as many edge cases as we could anticipate.
Now with the multi-modal capabilities of these models we can leverage the language capabilities along side vision to extract structured data from a table that has 'roughly this shape' and 'this location'.
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#160Earlier quoted context omitted.
I think its worth noting that if you are paying for electricity Local LLM is NOT free. In most cases you will find that Haiku is cheaper, faster, and better than anything that will run on your local machine.
If you need the heating then it is basically free.
Only if you use resistive electric heating, which is usually the most expensive heating available.