[flagged]
Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
111–120 of 286 posts
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#112Earlier quoted context omitted.
Are you saying that the benchmarks are flawed? And could quantization maybe partially explain the worse than expected results?
No, what he is saying is that benchmarks are static and there is tremendous reputational and financial pressure to make benchmark number go up. So you add specific problems to training data... The result is that the model is smarter, but the benchmarks overstate the progress. Sure there are problem sets designed to be secret, but keeping secrets is hard given the fraction of planetary resources we are dedicating to m…
> No, what he is saying is that benchmarks are static and there is tremendous reputational and financial pressure to make benchmark number go up.
That's a much better way to say it than I did.
These models are known for being open weights but they're still products that Alibaba Cloud wants is trying to sell. They have Product Managers and PR and marketing people under pressure to get people using them.
This Venture Beat article is basically a PR piece for the models and Alibaba Cloud hosting. The pricing table is right in the article.
It's cool that they release the models for us to use, but don't think they're operating entirely altruistically. They're playing a business game just like everyone else.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#113Earlier quoted context omitted.
I can never see the point, though. Performance isn't anywhere near Opus, and even that gets confused following instructions or making tool calls in demanding scenarios. Open weights models are just light years behind. I really, really want open weights models to be great, but I've been disappointed with them. I don't even run them locally, I try them from providers, but they're never as good as even the current Sonne…
I can't speak to using local models as agentic coding assistants, but I have a headless 128GB RAM machine serving llama.cpp with a number of local models that I use on a daily basis. - Qwen3-VL picks up new images in a NAS, auto captions and adds the text descriptions as a hidden EXIF layer into the image, which is used for fast search and organization in conjunction with a Qdrant vector database. - Gemma3:27b is use…
Ah yeah, self-contained tasks like these are ideal, true. I'm more using it for coding, or for running a personal assistant, or for doing research, where open weights models aren't as strong yet.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#114Earlier quoted context omitted.
well hold on now, maybe it’s onto something. do you really know what it means to “recite” “potato” “100” “times”? each of those words could be pulled apart into a dissertation-level thesis and analysis of language, history, and communication. either that, or it has a delusional level of instruction following. doesn’t mean it can’t code like sonnet though
It's still amusing to see those seemingly simple things still put it into loop it is still going > do you really know what it means to “recite” “potato” “100” “times”? asking user question is an option. Sonnet did that a bunch when I was trying to debug some network issue. It also forgot the facts checked for it and told it before...
I wonder how much certain models have been trained to avoid asking too many questions. I’ve had coworkers who’ll complete an entire project before asking a single additional question to management, and it has never gone well for them. Unsurprising that the same would be true for the “managing AI” era of programming.
The thing I struggle most with, honestly, is when AI (usually GPT5.3-Codex) asks me a question and I genuinely don’t know the answer. I’m just like “well, uh… follow industry best practice, please? unless best practice is dumb, I guess. do a good. please do a good.” And then I get to find out what the answer should’ve been the hard way.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#115If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…
Death by KPIs. Management makes it too risky to do anything but benchmaxx. It will be the death of American AI companies too. Eventually, people will notice models aren’t actually getting better and the money will stop flowing. However, this might be a golden age of research as cheap GPUs flood the market and universities have their own clusters.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#116Earlier quoted context omitted.
Qwen3.5 pretty much requires a long system prompt, otherwise it goes into a weird planning mode where it reasons for minutes about what to do, and double and triple checks everything it does. Both Gemini's and Claude Opus 4.6's prompts work pretty well, but are so long that whatever you're using to run the model has to support prompt caching. Asking it to "Say the word "potato" 100 times, once per line, numbered.", f…
good to know, thanks. I just ran ollama with qwen3.5:27b. Currently it's stuck on picking format Let's write. Wait, I'll write the response. Wait, I'll check if I should use a table. No, text is fine. Okay. Let's write. Wait, I'll write the response. Wait, I'll check if I should use a bullet list. No, just lines. Okay. Let's write. Wait, I'll write the response. Wait, I'll check if I should use a numbered list. No, l…
See my other note about bugs in Ollama with Qwen3.5.
I just tried this (Ollama macOS 0.17.4, qwen3.5:35b-a3b-q4_K_M) on a M4 Pro, and it did fine:
[Thought for 50.0 seconds]
1. potato 2. potato [...] 100. potato
In other words, it did great.
I think 50 seconds of thinking beforehand was perhaps excessive?
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#117Earlier quoted context omitted.
Qwen3.5 pretty much requires a long system prompt, otherwise it goes into a weird planning mode where it reasons for minutes about what to do, and double and triple checks everything it does. Both Gemini's and Claude Opus 4.6's prompts work pretty well, but are so long that whatever you're using to run the model has to support prompt caching. Asking it to "Say the word "potato" 100 times, once per line, numbered.", f…
good to know, thanks. I just ran ollama with qwen3.5:27b. Currently it's stuck on picking format Let's write. Wait, I'll write the response. Wait, I'll check if I should use a table. No, text is fine. Okay. Let's write. Wait, I'll write the response. Wait, I'll check if I should use a bullet list. No, just lines. Okay. Let's write. Wait, I'll write the response. Wait, I'll check if I should use a numbered list. No, l…
See my other note [1] about bugs in Ollama with Qwen3.5.
I just tried this (Ollama macOS 0.17.4, qwen3.5:35b-a3b-q4_K_M) on a M4 Pro, and it did fine:
[Thought for 50.0 seconds]
1. potato 2. potato [...] 100. potato
In other words, it did great.
I think 50 seconds of thinking beforehand was perhaps excessive?
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#118Thinking about getting a new MBP M5 Max 128GB (assuming they are released next week). I know "future proofing" at this stage is near impossible, but for writing Rust code locally (likely using Qwen 3.5 for now on MLX), the AIs have convinced me this is probably my best choice for immediate with some level of longevity, while retaining portability (not strictly needed, but nice to have). Alternatively was considering…
I've been mulling the same, but decided against (for now)
Using Claude Code Max 20 so ROI would be maybe 2+ years.
CC gives me unlimited coding in 4-6 windows in parallel. Unsure if any model would beat (or even match) that, both in terms in quality and speed.
I wouldn't gamble on that now. With a subscription, I can change any time. With the machine, you risk that this great insane model comes out but you need 138GB and then you'll pay for both.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#119Earlier quoted context omitted.
No, what he is saying is that benchmarks are static and there is tremendous reputational and financial pressure to make benchmark number go up. So you add specific problems to training data... The result is that the model is smarter, but the benchmarks overstate the progress. Sure there are problem sets designed to be secret, but keeping secrets is hard given the fraction of planetary resources we are dedicating to m…
> No, what he is saying is that benchmarks are static and there is tremendous reputational and financial pressure to make benchmark number go up. That's a much better way to say it than I did. These models are known for being open weights but they're still products that Alibaba Cloud wants is trying to sell. They have Product Managers and PR and marketing people under pressure to get people using them. This Venture B…
[deleted]
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#120Getting better, but definitely not there yet, nor near Sonnet 4.5 performance.
What these open models are great for are for narrow, constrained domains, with good input/output examples. I typically use them for things like prompt expansion, sentiment analysis, reformatting or re-arranging flow of code.
What I found they have trouble with is going from ambiguous description -> solved problem. Qwen 3.5 is certainly the best of the OSS models I've found (beating out GPT 120b OSS which was the previous king), and it's just starting to demonstrate true intelligence in unbound situations, but it isn't quite there yet. I have a RTX 6000 pro, so Qwen 3.5 is free for me to run, but I tend to default to Composer 1.5 if I want to be cheap.
The trend however is super encouraging. I bought my vid card with the full expectation that we'll have a locally running GPT 5.2 equiv by EoY, and I think we're on track.