Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

venturebeat.com

Post reply on HN
Print view View original

211–220 of 286 posts

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#211

Post by mstaoru »

__mharrison__ wrote: post #86
mstaoru wrote: post #53

I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…

Were you using mlx-lm? I've had good performance with that on Macs. (Sadly, the lead developer just left Apple.) Admittedly, I haven't tried these models on my Mac, but I have on my DGX Spark, and they ran fine. I didn't see the slowdown you're mentioning.

(I think) yes, via the latest openwebui + ollama.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#212

Post by mstaoru »

holoduke wrote: post #198
mstaoru wrote: post #53

I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…

Your Gemini or Opus question got send to a Texas datacenter where it got queued and processed by a subunit of 80 h200 140gb 1000w cards running a many billion or trillion parameter model. It took less that 200ms to process a single request. Your Claude cliënt decided to spawn 30 sub agents and iterated in a total of 90 requests totalling about 45000ms. Now compare that to your 100b transistor cpu doing something simi…

Right, it was more of a rhetorical question :) With my point being - how are these local models really useful to me now? Is the Only Way ™ to sell my house and build a 8x5090 monster?.. How does that compare to $20/month Opus? (Privacy aside.)

The second order thought from this is... will we get a value-based price leveling soon? If the alternative to a hosted LLM is to build $10-20k+ machine with $500+ monthly energy bills, will hosted price asymptotically climb up to reflect this reality?

Something to think about.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#213

Post by acchow »

CapsAdmin wrote: post #204
copperx wrote: post #186

Earlier quoted context omitted.

Are you using stepfun mostly because it's free, or is it better than other models at some things?

I pay for copilot to access anthropic, google and openai models. Claude code always give me rate limits. Claude through copilot is a bit slow, but copilot has constant network request issues or something, but at least I don't get rate limited as often. At least local models always work, is faster (50+ tps with qwen3.5 35b a4b on a 4090) and most importantly never hit a rate limit.

> Claude code always give me rate limits

> 50+ tps with qwen3.5 35b a4b on a 4090

But qwen3.5 35b is worse than even Claude Haiku 4.5. You could switch your Claude Code to use Haiku and never hit rate limits. Also gets similar 50tps.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#214

Post by etyhhgfff »

chr15m wrote: post #187
shell0x wrote: post #172

Earlier quoted context omitted.

What's the problem with Chinese models? The models are already open which makes them more trustworthy than the American closed models.

They are trained to respond to certain topics in a way that does not align with real world evidence. Pretty much the opposite of what you want in such a tool. This is trivial to test and verify yourself. Just pick any topic you think has a chance of being censored. You can do the same on American models and compare results.

I strongly agree, that this is an issue that needs to be adressed, but for everyday coding tasks it wont matter in 99.9 percent of cases.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#215

Post by baq »

Aurornis wrote: post #102

If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…

they're distilling claude and openai obviously.

that said, sonnet 4.5 is not a good model today, March 1st 2026. (it blew my mind on its release day, September 29th, 2025.)

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#216

Post by etyhhgfff »

mstaoru wrote: post #212
holoduke wrote: post #198

Earlier quoted context omitted.

Your Gemini or Opus question got send to a Texas datacenter where it got queued and processed by a subunit of 80 h200 140gb 1000w cards running a many billion or trillion parameter model. It took less that 200ms to process a single request. Your Claude cliënt decided to spawn 30 sub agents and iterated in a total of 90 requests totalling about 45000ms. Now compare that to your 100b transistor cpu doing something simi…

Right, it was more of a rhetorical question :) With my point being - how are these local models really useful to me now? Is the Only Way ™ to sell my house and build a 8x5090 monster?.. How does that compare to $20/month Opus? (Privacy aside.) The second order thought from this is... will we get a value-based price leveling soon? If the alternative to a hosted LLM is to build $10-20k+ machine with $500+ monthly energ…

We would need a super high end AI accelerator with specialised cooling for less than 3k bucks to make it happen. Consumer gaming graphics card wont fit the bill. Problem is all TSMC capacity is already booked for years to come by the big players to build data center grade hardware with price tags and setup requirements out of consumer reach.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#217

Post by BoredomIsFun »

vlovich123 wrote: post #79
lm28469 wrote: post #69

Earlier quoted context omitted.

> Wonder what am I doing wrong? You're comparing 100b parameters open models running on a consumer laptop VS private models with at the very least 1t parameters running on racks of bleeding edge professional gpus Local agentic coding is closer to "shit me the boiler plate for an android app" not "deep research questions", especially on your machine

The hardware difference explains runtime performance differences, not task performance. Speculation is that the frontier models are all below 200B parameters but a 2x size difference wouldn’t fully explain task performance differences

Certainly not Opus. That beast feels very heavy - the coherence of longer form prose is usually a good marker, and it is able to spit 4000 words coherent short stories from a single shot.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#218

Post by BoredomIsFun »

827a wrote: post #175
vlovich123 wrote: post #79

Earlier quoted context omitted.

The hardware difference explains runtime performance differences, not task performance. Speculation is that the frontier models are all below 200B parameters but a 2x size difference wouldn’t fully explain task performance differences

He's running a 35B parameter model. Frontier models are well over a trillion parameters at this point. Parameters = smarts. There are 1T+ open source models (e.g. GLM5), and they're actually getting to the point of being comparable with the closed source models; but you cannot remotely run them on any hardware available to us. Core speed/count and memory bandwidth determines your performance. Memory size determines y…

> There are 1T+ open source models (e.g. GLM5),

GLM-5 is ~750B model.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#219

Post by BoredomIsFun »

suprjami wrote: post #18
rahimnathwani wrote: post #15

Earlier quoted context omitted.

There seem to be a lot of different Q4s of this model: https://www.reddit.com/r/LocalLLaMA/s/kHUnFWZXom I'm curious which one you're using.

Unsloth Dynamic. Don't bother with anything else.

FYI UD quants of 3.5-35BA3B are broken, use bartowski or AesSedai ones.
Post reply on HN