Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

venturebeat.com

Post reply on HN
Print view View original

21–30 of 286 posts

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#21

Post by msuniverse2026 »

xienze wrote: post #10
erelong wrote: post #9

What kind of hardware does HN recommend or like to run these models?

It's less than you'd think. I'm using the 35B-A3B model on an A5000, which is something like a slightly faster 3080 with 24GB VRAM. I'm able to fit the entire Q4 model in memory with 128K context (and I think I would probably be able to do 256K since I still have like 4GB of VRAM free). The prompt processing is something like 1K tokens/second and generates around 100 tokens/second. Plenty fast for agentic use via Ope…

I've had an AMD card for the last 5 years, so I kinda just tuned out of local LLM releases because AMD seemed to abandon rocm for my card (6900xt) - Is AMD capable of anything these days?

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#22

Post by pityJuke »

CharlesW wrote: post #11
lostmsu wrote: post #6

Earlier quoted context omitted.

Not exactly, but pretty close: https://artificialanalysis.ai/models/capabilities/coding?mod... Somewhere between Haiku 4.5 and Sonnet 4.5

> Somewhere between Haiku 4.5 and Sonnet 4.5 That's like saying "somewhere between Eliza and Haiku 4.5". Haiku is not even a so-called 'reasoning model'.¹ ¹ To preempt the easily-offended, this is what the latest Opus 4.6 in today's Claude Code update says: "Claude Haiku 4.5 is not a reasoning model — it's optimized for speed and cost efficiency. It's the fastest model in the Claude family, good for quick, straightfo…

Haiku 4.5 is a reasoning model. [0]

[0]: https://www-cdn.anthropic.com/7aad69bf12627d42234e01ee7c3630...

> Claude Haiku 4.5, a new hybrid reasoning large language model from Anthropic in our small, fast model class.

> As with each model released by Anthropic beginning with Claude Sonnet 3.7, Claude Haiku 4.5 is a hybrid reasoning model. This means that by default the model will answer a query rapidly, but users have the option to toggle on “extended thinking mode”, where the model will spend more time considering its response before it answers. Note that our previous model in the Haiku small-model class, Claude Haiku 3.5, did not have an extended thinking mode.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#23

Post by andsoitis »

erelong wrote: post #9

What kind of hardware does HN recommend or like to run these models?

For fast inference, you’d be hard pressed to beat an Nvidia RTX 5090 GPU.

Check out the HP Omen 45L Max: https://www.hp.com/us-en/shop/pdp/omen-max-45l-gaming-dt-gt2...

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#24

Post by andsoitis »

u1hcw9nx wrote: post #17
ramon156 wrote: post #7

Earlier quoted context omitted.

Ironically, chinese models so far have been less lobotomized compared to OAI and Anthropic's models

Qwen has been broadly aligned to give positive messages about China in English. https://chinamediaproject.org/2026/02/09/tokens-of-ai-bias/ An Analysis of Chinese LLM Censorship and Bias with Qwen 2 Instruct https://huggingface.co/blog/leonardlin/chinese-llm-censorshi...

Does it matter for your work?

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#26

Post by kristianpaul »

johnnyApplePRNG wrote: post #20
mark_l_watson wrote: post #8

The new 35b model is great. That said, it has slight incompatibility's with Claude Code. It is very good for tool use.

Claude code is designed for anthropic models. Try it with opencode!

Or Pi

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#27

Post by alexpotato »

I recently wrote a guide on getting:

- llama.cpp

- OpenCode

- Qwen3-Coder-30B-A3B-Instruct in GGUF format (Q4_K_M quantization)

working on a M1 MacBook Pro (e.g. using brew).

It was bit finicky to get all of the pieces together so hopefully this can be used with these newer models.

https://gist.github.com/alexpotato/5b76989c24593962898294038...

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#28

Post by CamperBob2 »

erelong wrote: post #9

What kind of hardware does HN recommend or like to run these models?

I think the 27B dense model at full precision and 122B MoE at 4- or 6-bit quantization are legitimate killer apps for the 96 GB RTX 6000 Pro Blackwell, if the budget supports it.

I imagine any 24 GB card can run the lower quants at a reasonable rate, though, and those are still very good models.

Big fan of Qwen 3.5. It actually delivers on some of the hype that the previous wave of open models never lived up to.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#29

Post by copperx »

alexpotato wrote: post #27

I recently wrote a guide on getting: - llama.cpp - OpenCode - Qwen3-Coder-30B-A3B-Instruct in GGUF format (Q4_K_M quantization) working on a M1 MacBook Pro (e.g. using brew). It was bit finicky to get all of the pieces together so hopefully this can be used with these newer models. https://gist.github.com/alexpotato/5b76989c24593962898294038...

How fast does it run on your M1?
Post reply on HN