Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

venturebeat.com

Post reply on HN
Print view View original

81–90 of 286 posts

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#81

Post by lachiflippi »

PunchyHamster wrote: post #76
lachiflippi wrote: post #56

Earlier quoted context omitted.

Qwen3.5 pretty much requires a long system prompt, otherwise it goes into a weird planning mode where it reasons for minutes about what to do, and double and triple checks everything it does. Both Gemini's and Claude Opus 4.6's prompts work pretty well, but are so long that whatever you're using to run the model has to support prompt caching. Asking it to "Say the word "potato" 100 times, once per line, numbered.", f…

good to know, thanks. I just ran ollama with qwen3.5:27b. Currently it's stuck on picking format Let's write. Wait, I'll write the response. Wait, I'll check if I should use a table. No, text is fine. Okay. Let's write. Wait, I'll write the response. Wait, I'll check if I should use a bullet list. No, just lines. Okay. Let's write. Wait, I'll write the response. Wait, I'll check if I should use a numbered list. No, l…

Yeah, it tends to get stuck in loops like that a lot with everything set to default. I wonder if they distilled Gemini at some point, I've seen that get stuck in a similar "I will now do [thing]. I am preparing to do [thing]. I will do it." failure mode as well a couple of times.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#82

Post by rubyn00bie »

tempest_ wrote: post #33
solarkraft wrote: post #14

Smells like hyperbole. A lot of people making such claims don’t seem to have continued real world experience with these models or seem to have very weird standards for what they consider usable. Up until relatively recently, while people had already long been making these claims, it came with the asterisks of „oh, but you can’t practically use more than a few K tokens of context“.

Qwen3-Coder-30B-A3B-Instruct is good I think for in line IDE integration or operating on small functions or library code but I dont think you will get too far with one shot feature implementation that people are currently doing with Claude or whatever.

I could be doing something wrong, but I have not had any success with one shot feature implementations for any of the current models. There are always weird quirks, undesired behaviors, bad practices, or just egregiously broken implementations. A week or so ago, I had instructed Claude to do something at compile-time and it instead burned a phenomenal amount of tokens before yeeting the most absurd, and convoluted, runtime implementation—- that didn’t even work. At work I use it (or Codex) for specific tasks, delegating specific steps of the feature implementation.

The more I use the cloud based frontier models, the more virtue I find in using local, open source/weights, models because they tend to create much simpler code. They require more direct interaction from me, but the end result tends to be less buggy, easier to refactor/clean up, and more precisely what I wanted. I am personally excited to try this new model out here shortly on my 5090. If read the article correctly, it sounds like even the quantized versions have a “million”[1] token context window.

And to note, I’m sure I could use the same interaction loop for Claude or GPT, but the local models are free (minus the power) to run.

[1] I’m a dubious it won’t shite itself at even 50% of that. But even 250k would be amazing for a local model when I “only” have 32GB of VRAM.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#83

Post by xenospn »

MarsIronPI wrote: post #42
xenospn wrote: post #4

Are there any non-Chinese open models that offer comparable performance?

I think you could look into Minstral. There's also GPT-OSS but I'm not sure how well it stacks up. What's your problem with Chinese LLMs?

Nothing personally - Our customers send us highly sensitive financial documents to process. Using a foreign model to process their data (or even just for local testing) will most likely result in a u-turn.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#84

Post by delaminator »

lm28469 wrote: post #69
mstaoru wrote: post #53

I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…

> Wonder what am I doing wrong? You're comparing 100b parameters open models running on a consumer laptop VS private models with at the very least 1t parameters running on racks of bleeding edge professional gpus Local agentic coding is closer to "shit me the boiler plate for an android app" not "deep research questions", especially on your machine

Looks at the headline: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#85

Post by jbellis »

this is bullshit with a kernel of truth.

none of the qwen 3.5 models are anywhere near sonnet 4.5 class, not even the largest 397b.

BUT 27b is the smartest local-sized model in the world by a wide wide margin. (35b is shit. fast shit, but shit.)

benchmarks are complete, publishing on Monday.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#86

Post by __mharrison__ »

mstaoru wrote: post #53

I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…

Were you using mlx-lm? I've had good performance with that on Macs. (Sadly, the lead developer just left Apple.)

Admittedly, I haven't tried these models on my Mac, but I have on my DGX Spark, and they ran fine. I didn't see the slowdown you're mentioning.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#87

Post by satvikpendem »

zozbot234 wrote: post #55
mstaoru wrote: post #53

I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…

Running local AI models on a laptop is a weird choice. The Mini and especially the Studio form factor will have better cooling, lower prices for comparable specs and a much higher ceiling in performance and memory capacity.

I can take a laptop on the train.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#88

Post by rienko »

mstaoru wrote: post #53

I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…

use a larger model like Qwen3.5-122B-A10B quantized to 4/5/6 bits depending on how much context you desire, MLX versions if you want best tok/s on Mac HW.

if you are able to run something like mlx-community/MiniMax-M2.5-3bit (~100gb), my guess if the results are much better than 35b-a3b.

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#89

Post by lm28469 »

delaminator wrote: post #84
lm28469 wrote: post #69

Earlier quoted context omitted.

> Wonder what am I doing wrong? You're comparing 100b parameters open models running on a consumer laptop VS private models with at the very least 1t parameters running on racks of bleeding edge professional gpus Local agentic coding is closer to "shit me the boiler plate for an android app" not "deep research questions", especially on your machine

Looks at the headline: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

Yes and Devstral 2 24b q4 is supposed to be 90% as good but it can't even reliably write to a file on my machine.

There are the benchmarks, the promises, and what everybody can try at home

Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers

#90

Post by __mharrison__ »

solarkraft wrote: post #14

Smells like hyperbole. A lot of people making such claims don’t seem to have continued real world experience with these models or seem to have very weird standards for what they consider usable. Up until relatively recently, while people had already long been making these claims, it came with the asterisks of „oh, but you can’t practically use more than a few K tokens of context“.

I used the 35b model to create a polars implementation of PCA (no sklearn or imports other than math and polars). In less than 10 minutes I had the code. This is impressive to me considering how poorly all models were with polars until very recently. (They always hallucinated pandas code.)
Post reply on HN