Earlier quoted context omitted.
So I can use this in claude code with `ollama run claude`?
https://sleepingrobots.com/dreams/stop-using-ollama/
Thank you, I had no idea ollama was so shady! I will start using llama.cpp directly.
Qwen3.6-35B-A3B: Agentic coding power, now open to all
441–450 of 563 posts
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#442Earlier quoted context omitted.
I just wanted to express gratitude to you guys, you do great work. However, it is a little annoying to have to redownload big models though and keeping up with the AI news and community sentiment is a full time job. I wish there was some mechanism somewhere (on your site or Huggingface or something) for displaying feedback or confidence in a model being "ready for general use" before kicking off 100+ GB model downloa…
Hey thanks - yes agreed - for now we do: 1. Split metadata into shard 0 for huge models so 10B is for chat template fixes - however sometimes fixes cause a recalculation of the imatrix, which means all quants have to be re-made 2. Add HF discussion posts on each model talking about what changed, and on our Reddit and Twitter 3. Hugging Face XET now has de-duplication downloading of shards, so generally redownloading…
If you would know - is this also why LM Studio and Ollama model downloads often fail with a signature mismatch error?
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#443I'm broadly curious how people are using these local models. Literally, how are they attaching harnesses to this and finding more value than just renting tokens from Anthropic of OpenAI?
There are really nice GUIs for LLMs - CherryStudio for example, can be used with local or cloud models. There are also web-UIs - just like the labs ones. And you can connect coding agents like Codex, Copilot or Pi to local coding agents - the support OpenAI compatible APIs. It's literally a terminal command to start serving the model locally and you can connect various things to it, like Codex.
[dead]
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#444I'm broadly curious how people are using these local models. Literally, how are they attaching harnesses to this and finding more value than just renting tokens from Anthropic of OpenAI?
Idk about everyone else, but I don’t want to rent tokens forever. I want a self hosted model that is completely private and can’t be monitored or adulterated without me knowing. I use both currently, but I am excited at the prospect of maybe not having to in the near to mid future. I’ve increasingly started self hosting everything in my home lately because I got tired of SAAS rug pulls and I don’t see why LLM’s shoul…
Exactly. Relying on external compute for professional work is a non-starter IMO.
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#445China won again in terms of openness
Ironic
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#446Qwen3.6 and Gemma4 have the same issue of never getting to the point and just getting stuck in never ending repeating thought loops. Qwen3.5 is still the best local model that works.
I think the hype around Qwen and even Gemma4 often floated for views/attention glosses over that these models have clear gaps behind what closed models offer. In short, it has its uses but it would/should not be the main driver. Will it get better, I'm sure of it, but there is too much hype and exaggeration over open source models, for one the hardware simply isn't enough at a price point where we can run something t…
Give it 6 months
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#447Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#448Earlier quoted context omitted.
> You won't like it, but the answer is Apple. Or strix halo. Seems rather over simplified. The different levels of quants, for Qwen3.6 it's 10GB to 38.5GB. Qwen supports a context length of 262,144 natively, but can be extended to 1,010,000 and of course the context length can always be shortened. Just use one of the calculators and you'll get much more useful number.
What Strix Halo system has unified memory? A quick google says it's just a static vram allocation in ram, not that CPU and GPU can actively share memory at runtime
> What Strix Halo system has unified memory?
All of them. The static VRAM allocation is tiny (512MB), most of the memory is unified
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#449More benchmaxxing I see. Too bad there’s no rig with 256gb unified ram for under $1000
taps the sign Unified Memory Is A Marketing Gimmeck. Industrial-Scale Inference Servers Do Not Use It.
On M5 Pro/Max the memory is actually just attached straight to the GPU die. CPU accesses memory through the die-to-die bridge. I don't see the difference between that and a pure GPU from a memory connectivity point of view.
Wrt inference servers: sure, it's not cost-effective to have such a huge CPU die and a bunch of media accelerators on the GPU die if you just care about raw compute for inference and training. Apple SoCs are not tuned for that market, nor do they sell into it. I'm not building a datacentre, I'm trying to run inference on my home hardware that I also want to use for other things.
Re: Qwen3.6-35B-A3B: Agentic coding power, now open to all
#450Earlier quoted context omitted.
> You won't like it, but the answer is Apple. Or strix halo. Seems rather over simplified. The different levels of quants, for Qwen3.6 it's 10GB to 38.5GB. Qwen supports a context length of 262,144 natively, but can be extended to 1,010,000 and of course the context length can always be shortened. Just use one of the calculators and you'll get much more useful number.
What Strix Halo system has unified memory? A quick google says it's just a static vram allocation in ram, not that CPU and GPU can actively share memory at runtime
All. Keep in mind strix != strix halo.
You can get tablets, laptops, and desktops. I think windows is more limited and might require static allocation of video memory, not because it's a separate pool, just because windows isn't as flexible.
With linux you can just select the lowest number in bios (usually 256 or 512MB) then let linux balance the needs of the CPU/GPU. So you could easily run a model that requires 96GB or more.