Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers • Hacker News Board

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#41

Post by gunalx » Sat, Feb 28, 2026, 10:35 PM UTC

qwen 3.5 is really decent. oOtside for some weird failures on some scaffolding with seemingly different trained tools.

Strong vision and reasoning performance, and the 35-a3b model run s pretty ok on a 16gb GPU with some CPU layers.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#42

Post by MarsIronPI » Sat, Feb 28, 2026, 10:36 PM UTC

xenospn wrote: post #4

Are there any non-Chinese open models that offer comparable performance?

I think you could look into Minstral. There's also GPT-OSS but I'm not sure how well it stacks up.

What's your problem with Chinese LLMs?

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#43

Post by rahimnathwani » Sat, Feb 28, 2026, 10:36 PM UTC

suprjami wrote: post #18

rahimnathwani wrote: post #15

Earlier quoted context omitted.

There seem to be a lot of different Q4s of this model: https://www.reddit.com/r/LocalLLaMA/s/kHUnFWZXom I'm curious which one you're using.

Unsloth Dynamic. Don't bother with anything else.

UD-Q4_K_XL?

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#44

Post by laweijfmvo » Sat, Feb 28, 2026, 10:37 PM UTC

andsoitis wrote: post #23

erelong wrote: post #9

What kind of hardware does HN recommend or like to run these models?

For fast inference, you’d be hard pressed to beat an Nvidia RTX 5090 GPU. Check out the HP Omen 45L Max: https://www.hp.com/us-en/shop/pdp/omen-max-45l-gaming-dt-gt2...

I never would have guessed that in 2026, data centers would be measured in Watts and desktop PCs measured in liters.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#45

Post by DetroitThrow » Sat, Feb 28, 2026, 10:38 PM UTC

CharlesW wrote: post #35

pityJuke wrote: post #22

Earlier quoted context omitted.

Haiku 4.5 is a reasoning model. [0] [0]: https://www-cdn.anthropic.com/7aad69bf12627d42234e01ee7c3630... > Claude Haiku 4.5, a new hybrid reasoning large language model from Anthropic in our small, fast model class. > As with each model released by Anthropic beginning with Claude Sonnet 3.7, Claude Haiku 4.5 is a hybrid reasoning model. This means that by default the model will answer a query rapidly, but users have…

Sure, marketing people gonna market. But Haiku's 'extended thinking' mode is very different than the reasoning capabilities of Sonnet or Opus. I would absolutely believe mar-ticles that Qwen has achieved Haiku 4.5 'extended thinking' levels of coding prowess.

>Sure, marketing people gonna market.

Oh HN never change.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#46

Post by kristianp » Sat, Feb 28, 2026, 10:41 PM UTC

kristianpaul wrote: post #37

https://unsloth.ai/docs/models/qwen3.5#qwen3.5-27b “ Qwen3.5-27B For this guide we will be utilizing Dynamic 4-bit which works great on a 18GB RAM”

18GB was an odd 3-channel one-off for the M3 Pros. I guess there's a bunch of them out there, but how slow would 27B be on it, due to not being an MOE model.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#47

Post by [deleted] » Sat, Feb 28, 2026, 10:45 PM UTC

DetroitThrow wrote: post #45

CharlesW wrote: post #35

Earlier quoted context omitted.

Sure, marketing people gonna market. But Haiku's 'extended thinking' mode is very different than the reasoning capabilities of Sonnet or Opus. I would absolutely believe mar-ticles that Qwen has achieved Haiku 4.5 'extended thinking' levels of coding prowess.

>Sure, marketing people gonna market. Oh HN never change.

[deleted]

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#48

Post by PunchyHamster » Sat, Feb 28, 2026, 10:51 PM UTC

I asked it to recite potato 100 times coz I wanted to benchmark speed of CPU vs GPU. It's on 150 line of planning. It recited the requested thing 4 times already and started drafting the 5th response.

...yeah I doubt it

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#49

Post by robby_w_g » Sat, Feb 28, 2026, 10:51 PM UTC

alexpotato wrote: post #27

I recently wrote a guide on getting: - llama.cpp - OpenCode - Qwen3-Coder-30B-A3B-Instruct in GGUF format (Q4_K_M quantization) working on a M1 MacBook Pro (e.g. using brew). It was bit finicky to get all of the pieces together so hopefully this can be used with these newer models. https://gist.github.com/alexpotato/5b76989c24593962898294038...

Does your MBP have 32 GB of ram? I’m waiting on a local model that can run decently on 16 GB

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#50

Post by freeone3000 » Sat, Feb 28, 2026, 10:52 PM UTC

alexpotato wrote: post #27

I recently wrote a guide on getting: - llama.cpp - OpenCode - Qwen3-Coder-30B-A3B-Instruct in GGUF format (Q4_K_M quantization) working on a M1 MacBook Pro (e.g. using brew). It was bit finicky to get all of the pieces together so hopefully this can be used with these newer models. https://gist.github.com/alexpotato/5b76989c24593962898294038...

We can also run LM Studio and get it installed with one search and one click, exposed through an OpenAI-compatible API.