Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

venturebeat.com

Post reply on HN
Print view View original

121–130 of 286 posts

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#121

Post by vunderba »

stavros wrote: post #113
vunderba wrote: post #105

Earlier quoted context omitted.

I can't speak to using local models as agentic coding assistants, but I have a headless 128GB RAM machine serving llama.cpp with a number of local models that I use on a daily basis. - Qwen3-VL picks up new images in a NAS, auto captions and adds the text descriptions as a hidden EXIF layer into the image, which is used for fast search and organization in conjunction with a Qdrant vector database. - Gemma3:27b is use…

Ah yeah, self-contained tasks like these are ideal, true. I'm more using it for coding, or for running a personal assistant, or for doing research, where open weights models aren't as strong yet.

Understood. Research would make me especially leery; I’d be afraid of losing any potential gains as I'd feel compelled to always go and validate its claims (though I suppose you could mitigate it a little bit with search engine tooling like Kagi's MCP system).

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#122

Post by wolvoleo »

Aurornis wrote: post #102

If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…

All models are doing that. Not only the open source ones.

I bet the cloud ones are doing it a lot more because they can also affect the runtime side which the open source ones can't.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#123

Post by eurekin »

Aurornis wrote: post #102

If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…

Very good point. I'm playing with them too and got to the same conclusion.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#124

Post by wolvoleo »

mstaoru wrote: post #53

I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…

Well first of all you're running a long intense task on a thermally constrained machine. Your MacBook Pro is optimised for portability and battery life, not max performance under load. And apple's obsession with thinness overrules thermal performance for them. Short peaks will be ok but a 45 minute task will thoroughly saturate the cooling system.

Even on servers this can happen. At work we have a 2U sized server with two 250W class GPUs. And I found that by pinning the case fans at 100% I can get 30% more performance out of GPU tasks which translates to several days faster for our usecase. It does mean I can literally hear the fans screaming in the hallway outside the equipment room but ok lol. Who cares. But a laptop just can't compare.

Something with a desktop GPU or even better something with HBM3 would run much better. Local models get slow when you use a ton of context and the memory bandwidth of a MacBook Pro while better than a pc is still not amazing.

And yeah the heaviest tasks are not great on local models. I tend to run the low hanging fruit locally and the stuff where I really need the best in the cloud. I don't agree local models are on par, however I don't think they really need to be for a lot of tasks.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#125

Post by MarsIronPI »

icase wrote: post #72
MarsIronPI wrote: post #42

Earlier quoted context omitted.

I think you could look into Minstral. There's also GPT-OSS but I'm not sure how well it stacks up. What's your problem with Chinese LLMs?

it’s not obvious to you why someone would want to avoid models created by our enemies?

No, it's not. They're just collections of numbers that can be harnessed to produce outputs. I check the outputs and if they're good I use them. If they're not, I ignore them and there's no harm done. Obviously I don't trust them to be accurate sources of information, but I don't trust American corporate LLMs much more.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#126

Post by bourjwahwah »

Aurornis wrote: post #102

If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…

[dead]

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#127

Post by MarsIronPI »

xenospn wrote: post #83
MarsIronPI wrote: post #42

Earlier quoted context omitted.

I think you could look into Minstral. There's also GPT-OSS but I'm not sure how well it stacks up. What's your problem with Chinese LLMs?

Nothing personally - Our customers send us highly sensitive financial documents to process. Using a foreign model to process their data (or even just for local testing) will most likely result in a u-turn.

What if you run them locally, or use a US-based provider that hosts them? IMO the provenance of the weights doesn't matter. You're right that the location of the hoster does, though.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#128

Post by kir-gadjello »

Aurornis wrote: post #102

If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…

Respectfully, from my experience and a few billions of tokens consumed, some opensource models really are strong and useful. Specifically StepFun-3.5-flash https://github.com/stepfun-ai/Step-3.5-Flash

I'm working on a pretty complex Rust codebase right now, with hundreds of integration tests and nontrivial concurrency, and stepfun powers through.

I have no relation to stepfun, and I'm saying this purely from deep respect to the team that managed to pack this performance in 196B/11B active envelope.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#129

Post by noosphr »

Aurornis wrote: post #102

If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…

It's not just the open source ones.

The only benchmarks worth anything are dynamic ones which can be scaled up.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#130

Post by aappleby »

kir-gadjello wrote: post #128
Aurornis wrote: post #102

If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…

Respectfully, from my experience and a few billions of tokens consumed, some opensource models really are strong and useful. Specifically StepFun-3.5-flash https://github.com/stepfun-ai/Step-3.5-Flash I'm working on a pretty complex Rust codebase right now, with hundreds of integration tests and nontrivial concurrency, and stepfun powers through. I have no relation to stepfun, and I'm saying this purely from deep res…

What are you running that model on?
Post reply on HN