Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

venturebeat.com

Post reply on HN
Print view View original

251–260 of 286 posts

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#251

Post by CharlesW »

DetroitThrow wrote: post #144
CharlesW wrote: post #71

Earlier quoted context omitted.

Not sure what this means, but as a marketing person myself, here's what happened: One day, an Anthropican involved in the Haiku 4.5 launch shrugged, weighed the odds of getting spanked for equating "extended thinking" with "reasoning", and then used Claude to generate copy declaring that. It's not rocket surgery!

It's mainly that people on here, regardless of profession, speak incorrectly but confidentally about things that could be easily verified with a Google search or basic familiarity with the thing in question. Haiku 4.5 is a reasoning model, regardless of whatever hallucination you read. Being a hybrid reasoning model means that, depending on the complexity of the question and whether you explicitly enable reasoning (t…

> It's mainly that people on here, regardless of profession, speak incorrectly but confidentally about things that could be easily verified with a Google search or basic familiarity with the thing in question.

Yep. And if your heart wants to call Haiku a "reasoning model", obviously you must listen. It doesn't meet that bar for me for a couple reasons: (1) It lacks both "adaptive thinking" and "interleaved thinking" (per Anthropic, both critical for reasoning models), and (2) it also performed unacceptably with a real-world collection of very basic reasoning tasks that I tried using it for.¹ I'm glad you're having better luck with it.

That said, it's a great and affordable little model for what it was designed for!

¹ I once made the mistake of converting a bunch of skills (which require basic reasoning) to use Haiku for Axiom (https://charleswiltgen.github.io/Axiom/). It failed miserably, and wow, did users let me have it. On the bright side, as a result I'm now far better at testing models' ability to reason.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#252

Post by joe_mamba »

u1hcw9nx wrote: post #232
joe_mamba wrote: post #54

Earlier quoted context omitted.

Impressive, very nice, now let's see what would be the odds that the US models developed in SV are also highly positive about Californian and Democrats politics.

Whataboutism https://en.wikipedia.org/wiki/Whataboutism

Screaming whataboutism is the only way people know to avoid answering an obvious fact. That LLMs have the biases of the governments where the came from doesn't matter if it's China, US, India, EU, etc

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#253

Post by Aurornis »

kir-gadjello wrote: post #128
Aurornis wrote: post #102

If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…

Respectfully, from my experience and a few billions of tokens consumed, some opensource models really are strong and useful. Specifically StepFun-3.5-flash https://github.com/stepfun-ai/Step-3.5-Flash I'm working on a pretty complex Rust codebase right now, with hundreds of integration tests and nontrivial concurrency, and stepfun powers through. I have no relation to stepfun, and I'm saying this purely from deep res…

> some opensource models really are strong and useful

To be clear I never said they weren’t strong or useful. I use them for some small tasks too.

I said they’re not equivalent to SOTA models from 6 months ago, which is what is always claimed.

Then it turns into a Motte and Bailey game where that argument is replaced with the simpler argument that they’re useful for open weights models. I’m not disagreeing with that part. I’m disagree with the first assertion that they’re equivalent to Sonnet 4.5

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#254

Post by Aurornis »

lend000 wrote: post #194
Aurornis wrote: post #102

If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…

Yes and no. "Last-gen" (like, from 6 months ago) frontier models do still tend to outperform the best open source models. But some models, especially GLM-5, really have captured whatever circuitry drives pattern matching in the models they were trained off of. I like this benchmark that competes models against one another in competitive environments, which seems like it can't really be gamed: https://gertlabs.com

> Yes and no. "Last-gen" (like, from 6 months ago) frontier models do still tend to outperform the best open source models

That’s exactly what I said, though. The headline we’re commenting under claims they’re Sonnet 4.5 level but they’re not.

I don’t disagree that they’re powerful for open models. I’m pointing out that anyone reading these headlines who expects a cheap or local Sonnet 4.5 is going to discover that it’s not true.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#255

Post by kir-gadjello »

Aurornis wrote: post #253
kir-gadjello wrote: post #128

Earlier quoted context omitted.

Respectfully, from my experience and a few billions of tokens consumed, some opensource models really are strong and useful. Specifically StepFun-3.5-flash https://github.com/stepfun-ai/Step-3.5-Flash I'm working on a pretty complex Rust codebase right now, with hundreds of integration tests and nontrivial concurrency, and stepfun powers through. I have no relation to stepfun, and I'm saying this purely from deep res…

> some opensource models really are strong and useful To be clear I never said they weren’t strong or useful. I use them for some small tasks too. I said they’re not equivalent to SOTA models from 6 months ago, which is what is always claimed. Then it turns into a Motte and Bailey game where that argument is replaced with the simpler argument that they’re useful for open weights models. I’m not disagreeing with that…

They are not equivalent 1:1, esp. in knowledge coverage (given OOM param size difference) and in taste (Sonnet wins, but for taste one can also use Kimi K2.5), but in my hardcore use (high-performance realtime simulations of various kinds) I would prefer StepFun-3.5-Flash to Sonnet 4 strongly and to 4.5 often enough without a decisive advantage in using exclusively Sonnet 4.5. For truly hard tasks or specifications I would turn to 5.2 or 5.3-codex of course - but one KPI for quality of my work as a lead engineer is to ensure that truly hard tasks are known, bounded and planned-for in advance.

Maybe my detailed, requirement-based/spec-based prompting style makes the difference between anthropic's and OSS models smaller and people just like how good Anthropic's models are at reading the programmer's intent from short concise prompts.

Frankly, I think the 1:1 equivalent is an impossible standard given the set of priorities and decisions frontier labs make when setting up their pre-, mid- and post-training pipelines, and benchmark-wise it is achievable for a smaller OSS model to align with Sonnet 4.5 even on hard benchmarks.

Given the relatively underwhelming Sonnet 4.5 benchmarks [1], I think StepFun might have an edge over it esp. in Math/STEM [2] - even an old deepseek-3.2 (not speciale!) had a similar aggregate score. With 4.6 Anthropic ofc vastly improved their benchmark game, and it now truly looks like a frontier model.

1. https://artificialanalysis.ai/models/claude-4-5-sonnet-think... 2. https://matharena.ai/models/stepfun_3_5_flash

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#256

Post by kir-gadjello »

jasonni wrote: post #208
kir-gadjello wrote: post #128

Earlier quoted context omitted.

Respectfully, from my experience and a few billions of tokens consumed, some opensource models really are strong and useful. Specifically StepFun-3.5-flash https://github.com/stepfun-ai/Step-3.5-Flash I'm working on a pretty complex Rust codebase right now, with hundreds of integration tests and nontrivial concurrency, and stepfun powers through. I have no relation to stepfun, and I'm saying this purely from deep res…

What coding agent do you use with StepFun-3.5-flash? I just tried it from siliconflow's api with opencode. The toolcalling is broken: AI_InvalidResponseDataError: Expected 'function.name' to be a string.

I use pi, but I'm almost done writing a better alternative that doesn't have pi's stability issues. 80K Rust SLOC and a few hundred tests btw.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#258

Post by zozbot234 »

mstaoru wrote: post #248
zozbot234 wrote: post #55

Earlier quoted context omitted.

Running local AI models on a laptop is a weird choice. The Mini and especially the Studio form factor will have better cooling, lower prices for comparable specs and a much higher ceiling in performance and memory capacity.

So it's back to the original question, why spend $5-10k on the Studio, when it will still be 10x slower and half the intelligence vs. $20 Sonnet?.. What is the point (besides privacy) to use local models now for coding? PS: I can understand that isolated "valuable" problems like sorting photo collection or feeding a cat via ESPHome can be solved with local models.

You can't use a $20 Sonnet subscription for general agentic use cases, you have to pay for API use on a per-token basis. The $20 and $200 subscriptions are widely considered unsustainable as such. If anything, the real competition is third-party cheap inference providers.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#259

Post by zozbot234 »

deepsquirrelnet wrote: post #153
jackcosgrove wrote: post #146

I am a total neophyte when it comes to LLMs, and only recently started poking around into the internals of them. The first thing that struck me was that float32 dimensions seemed very generous . I then discovered what quantization is by reading a blog post about binary quantization. That seemed too good to be true. I asked Claude to design an analysis assessing the fidelity of 1, 2, 4, and 8 bit quantization. Claude…

4-bit quantization on newer nvidia hardware is being supported in training as well these days. I believe the gpt-oss models were trained natively in MXFP4, which is a 4-bit floating point / e2m1 (2-exponent, 1 bit mantissa, 1 bit sign). It doesn't seem terribly common yet though. I think it is challenging to keep it stable. [1] https://www.opencompute.org/blog/amd-arm-intel-meta-microsof... [2] https://www.opencomput…

mxfp4 is a block-based floating point format. The E2M1 format applies to individual values, but each 32-values block also has a shared 8-bit floating point exponent to provide scaling information about the whole block.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#260

Post by Aerroon »

kir-gadjello wrote: post #131
aappleby wrote: post #130

Earlier quoted context omitted.

What are you running that model on?

I just use openrouter, it's free for now. But I would pay 30-100$ to use it 24/7.

Have you tried Minimax M2.5? How did it compare?
Post reply on HN