Smells like hyperbole. A lot of people making such claims don’t seem to have continued real world experience with these models or seem to have very weird standards for what they consider usable. Up until relatively recently, while people had already long been making these claims, it came with the asterisks of „oh, but you can’t practically use more than a few K tokens of context“.
Qwen3-Coder-30B-A3B-Instruct is good I think for in line IDE integration or operating on small functions or library code but I dont think you will get too far with one shot feature implementation that people are currently doing with Claude or whatever.
Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
261–270 of 286 posts
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#262Earlier quoted context omitted.
So it's back to the original question, why spend $5-10k on the Studio, when it will still be 10x slower and half the intelligence vs. $20 Sonnet?.. What is the point (besides privacy) to use local models now for coding? PS: I can understand that isolated "valuable" problems like sorting photo collection or feeding a cat via ESPHome can be solved with local models.
You can't use a $20 Sonnet subscription for general agentic use cases, you have to pay for API use on a per-token basis. The $20 and $200 subscriptions are widely considered unsustainable as such. If anything, the real competition is third-party cheap inference providers.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#263Earlier quoted context omitted.
Try the 27B dense model. It will likely do much better than the 35b MoE with only 3B active experts. Also, performance on research-y questions isn't always a good indicator of how the model will do for code generation or agent orchestration.
Currently sat waiting for the unsloth fixed quants to drop, but I'm on the edge of my seat for this.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#264Earlier quoted context omitted.
Running local AI models on a laptop is a weird choice. The Mini and especially the Studio form factor will have better cooling, lower prices for comparable specs and a much higher ceiling in performance and memory capacity.
So it's back to the original question, why spend $5-10k on the Studio, when it will still be 10x slower and half the intelligence vs. $20 Sonnet?.. What is the point (besides privacy) to use local models now for coding? PS: I can understand that isolated "valuable" problems like sorting photo collection or feeding a cat via ESPHome can be solved with local models.
Are the LLMs very useful? That is a whole other discussion...
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#265Earlier quoted context omitted.
What coding agent do you use with StepFun-3.5-flash? I just tried it from siliconflow's api with opencode. The toolcalling is broken: AI_InvalidResponseDataError: Expected 'function.name' to be a string.
I use pi, but I'm almost done writing a better alternative that doesn't have pi's stability issues. 80K Rust SLOC and a few hundred tests btw.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#266I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…
The biggest gaps are not in hardware or model size. There is a lot of logical fallacy in the industry. Most people believe bigger is better. For model size, compute, tools, etc. The reality in ML is that small models can perform better at a narrow problem set than large ones. The key is the narrow problem set. Opus can write you a poem, create a shopping list, and analyze your massive code base. We trained our model…
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#267Earlier quoted context omitted.
Claude Sonnet can easily one-shot that without specifically asking for plan first.
I believe you, but performance on 10-word prompts is pretty useless as a metric
I build micro apps from 10-word prompts multiple times a day.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#268Earlier quoted context omitted.
I use pi, but I'm almost done writing a better alternative that doesn't have pi's stability issues. 80K Rust SLOC and a few hundred tests btw.
Any place we can look for you to release this?
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#269Earlier quoted context omitted.
I have a pair of Radeon AI PRO R9700 with 32Gb, and so far they have been a pleasure to use. Drivers work out-of-the-box, and they are completely quiet when unused. They are capped at 300W power, so even at 100% utilization they are not too loud. I was thinking about adding after-market liquid cooling for them, but they're fine without it.
This is great to hear! Out of curiosity, which brand did you go with? I tend to stick to Sapphire but the prices are within $200 of each other.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#270Earlier quoted context omitted.
Unsloth Dynamic. Don't bother with anything else.
For anyone else trying to run this on a Mac with 32GB unified RAM, this is what worked for me: First, make sure enough memory is allocated to the gpu: sudo sysctl -w iogpu.wired_limit_mb=24000 Then run llama.cpp but reduce RAM needs by limiting the context window and turning off vision support. (And turn off reasoning for now as it's not needed for simple queries.) llama-server \ -hf unsloth/Qwen3.5-35B-A3B-GGUF:UD-Q…
Many user benchmarks report up to 30% better memory usage and up to 50% higher token generation speed:
https://reddit.com/r/LocalLLaMA/comments/1fz6z79/lm_studio_s...
As the post says, LM Studio has an MLX backend which makes it easy to use.
If you still want to stick with llama-server and GGUF, look at llama-swap which allows you to run one frontend which provides a list of models and dynamically starts a llama-server process with the right model:
https://github.com/mostlygeek/llama-swap
(actually you could run any OpenAI-compatible server process with llama-swap)