Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

venturebeat.com

Post reply on HN
Print view View original

11–20 of 286 posts

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#11

Post by CharlesW »

lostmsu wrote: post #6
aliljet wrote: post #5

Is this actually true? I want to see actual evals that match this up with Sonnet 4.5.

Not exactly, but pretty close: https://artificialanalysis.ai/models/capabilities/coding?mod... Somewhere between Haiku 4.5 and Sonnet 4.5

> Somewhere between Haiku 4.5 and Sonnet 4.5

That's like saying "somewhere between Eliza and Haiku 4.5". Haiku is not even a so-called 'reasoning model'.¹

¹ To preempt the easily-offended, this is what the latest Opus 4.6 in today's Claude Code update says: "Claude Haiku 4.5 is not a reasoning model — it's optimized for speed and cost efficiency. It's the fastest model in the Claude family, good for quick, straightforward tasks, but it doesn't have extended thinking/reasoning capabilities."

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#12

Post by suprjami »

erelong wrote: post #9

What kind of hardware does HN recommend or like to run these models?

The cheapest option is two 3060 12G cards. You'll be able to fit the Q4 of the 27B or 35B with an okay context window.

If you want to spend twice as much for more speed, get a 3090/4090/5090.

If you want long context, get two of them.

If you have enough spare cash to buy a car, get an RTX Ada with 96G VRAM.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#13

Post by solarkraft »

What are the recommended 4 bit quants for the 35B model? I don’t see official ones: https://huggingface.co/models?other=base_model:quantized:Qwe...

Edit: The unsloth quants seem to have been fixed, so they are probably the go-to again: https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#14

Post by solarkraft »

Smells like hyperbole. A lot of people making such claims don’t seem to have continued real world experience with these models or seem to have very weird standards for what they consider usable.

Up until relatively recently, while people had already long been making these claims, it came with the asterisks of „oh, but you can’t practically use more than a few K tokens of context“.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#15

Post by rahimnathwani »

xienze wrote: post #10
erelong wrote: post #9

What kind of hardware does HN recommend or like to run these models?

It's less than you'd think. I'm using the 35B-A3B model on an A5000, which is something like a slightly faster 3080 with 24GB VRAM. I'm able to fit the entire Q4 model in memory with 128K context (and I think I would probably be able to do 256K since I still have like 4GB of VRAM free). The prompt processing is something like 1K tokens/second and generates around 100 tokens/second. Plenty fast for agentic use via Ope…

There seem to be a lot of different Q4s of this model: https://www.reddit.com/r/LocalLLaMA/s/kHUnFWZXom

I'm curious which one you're using.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#16

Post by pinum »

lostmsu wrote: post #6
aliljet wrote: post #5

Is this actually true? I want to see actual evals that match this up with Sonnet 4.5.

Not exactly, but pretty close: https://artificialanalysis.ai/models/capabilities/coding?mod... Somewhere between Haiku 4.5 and Sonnet 4.5

Looks much closer to Haiku than Sonnet.

Maybe "Qwen3.5 122B offers Haiku 4.5 performance on local computers" would be a more realistic and defensible claim.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#17

Post by u1hcw9nx »

ramon156 wrote: post #7
u1hcw9nx wrote: post #2

[flagged]

Ironically, chinese models so far have been less lobotomized compared to OAI and Anthropic's models

Qwen has been broadly aligned to give positive messages about China in English. https://chinamediaproject.org/2026/02/09/tokens-of-ai-bias/

An Analysis of Chinese LLM Censorship and Bias with Qwen 2 Instruct https://huggingface.co/blog/leonardlin/chinese-llm-censorshi...

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#18

Post by suprjami »

rahimnathwani wrote: post #15
xienze wrote: post #10

Earlier quoted context omitted.

It's less than you'd think. I'm using the 35B-A3B model on an A5000, which is something like a slightly faster 3080 with 24GB VRAM. I'm able to fit the entire Q4 model in memory with 128K context (and I think I would probably be able to do 256K since I still have like 4GB of VRAM free). The prompt processing is something like 1K tokens/second and generates around 100 tokens/second. Plenty fast for agentic use via Ope…

There seem to be a lot of different Q4s of this model: https://www.reddit.com/r/LocalLLaMA/s/kHUnFWZXom I'm curious which one you're using.

Unsloth Dynamic. Don't bother with anything else.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#19

Post by dajonker »

erelong wrote: post #9

What kind of hardware does HN recommend or like to run these models?

Radeon R9700 with 32 GB VRAM is relatively affordable for the amount of RAM and with llama.cpp it runs fast enough for most things. These are workstation cards with blower fans and they are LOUD. Otherwise if you have the money to burn get a 5090 for speeeed and relatively low noise, especially if you limit power usage.
Post reply on HN