Can someone who has done this, simplify and say what specs we need on a `local computer` to run and test this, with a reasonable speed?
Excluding MBP M5 128GB.
Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
201–210 of 286 posts
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#202I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…
Depending on the specificity of the research, having a model with fewer parameters will come with a higher penalty. If you want a model to perform better at something specific while staying smaller, generally it will take specific training to achieve that.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#203Earlier quoted context omitted.
Respectfully, from my experience and a few billions of tokens consumed, some opensource models really are strong and useful. Specifically StepFun-3.5-flash https://github.com/stepfun-ai/Step-3.5-Flash I'm working on a pretty complex Rust codebase right now, with hundreds of integration tests and nontrivial concurrency, and stepfun powers through. I have no relation to stepfun, and I'm saying this purely from deep res…
Are you using stepfun mostly because it's free, or is it better than other models at some things?
I think we are at this point where the hard ceiling of a strong model is pretty hard to delineate reliably (at least in coding, in research work it's clearer ofc) - and in a good sense, meaning with suitable task decomposition or a test harness or a good abstraction you can make the model do what you thought it could not. StepFun is a strong model and I really enjoyed studying and comparing it to others by coding pretty complex projects semi-autonomously (will do a write up on this soon tm).
Even purely pragmatically, StepFun covers 95% of my research+SWE coding needs, and for the remaining 5% I can access the large frontier models. I was surprised StepFun is even decent at planning and research, so it is possible to get by with it and nothing else (1), but ofc for minmaxing the best frontier model is still the best planner (although the latest deepseek is surprisingly good too).
Finally we are at a point where there is a clear separation of labor between frontier & strong+fast models, but tbh shoehorning StepFun into this "strong+fast" category feels limiting, I think it has greater potential.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#204Earlier quoted context omitted.
Respectfully, from my experience and a few billions of tokens consumed, some opensource models really are strong and useful. Specifically StepFun-3.5-flash https://github.com/stepfun-ai/Step-3.5-Flash I'm working on a pretty complex Rust codebase right now, with hundreds of integration tests and nontrivial concurrency, and stepfun powers through. I have no relation to stepfun, and I'm saying this purely from deep res…
Are you using stepfun mostly because it's free, or is it better than other models at some things?
I pay for copilot to access anthropic, google and openai models.
Claude code always give me rate limits. Claude through copilot is a bit slow, but copilot has constant network request issues or something, but at least I don't get rate limited as often.
At least local models always work, is faster (50+ tps with qwen3.5 35b a4b on a 4090) and most importantly never hit a rate limit.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#205Anyone have recommendations on EU services where one could run open models before buying expensive hardware?
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#206Can someone who has done this, simplify and say what specs we need on a `local computer` to run and test this, with a reasonable speed? Excluding MBP M5 128GB.
It is slow but usable via opencode on a mbp m3 max 48 gb. So I guess hosted is still the better option for most people.
The local models are considerably better relative to the hosted ones compared to 6 months ago. Bench maxing or not - stuff is happening in this area for sure.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#207Anyone have recommendations on EU services where one could run open models before buying expensive hardware?
Koyeb (recently acquired by Mistral if I'm not mistaken) have GPUs you can rent by the minute and they also have one-click deploy of some open models.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#208If you're new to this: All of the open source models are playing benchmark optimization games. Every new open weight model comes with promises of being as good as something SOTA from a few months ago then they always disappoint in actual use. I've been playing with Qwen3-Coder-Next and the Qwen3.5 models since they were each released. They are impressive, but they are not performing at Sonnet 4.5 level in my experien…
Respectfully, from my experience and a few billions of tokens consumed, some opensource models really are strong and useful. Specifically StepFun-3.5-flash https://github.com/stepfun-ai/Step-3.5-Flash I'm working on a pretty complex Rust codebase right now, with hundreds of integration tests and nontrivial concurrency, and stepfun powers through. I have no relation to stepfun, and I'm saying this purely from deep res…
What coding agent do you use with StepFun-3.5-flash? I just tried it from siliconflow's api with opencode. The toolcalling is broken:
AI_InvalidResponseDataError: Expected 'function.name' to be a string.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#209I decided to try Qwen3.5 122B in LM Studio with Opencode and I am impressed. It's not super slow (M4 Max/128GB) and it's pretty close to how Claude Code feels. Getting pretty good code analysis, definitely feels Sonnet-esque. I'm hyped completely local alternatives are getting so good.
Re: Qwen3.5 122B and 35B models offer Sonnet 4.5 performance on local computers
#210Earlier quoted context omitted.
> Wonder what am I doing wrong? You're comparing 100b parameters open models running on a consumer laptop VS private models with at the very least 1t parameters running on racks of bleeding edge professional gpus Local agentic coding is closer to "shit me the boiler plate for an android app" not "deep research questions", especially on your machine
I'll add, AI Labs put a lot of resources into allowing the AI to search the web.. that makes a big difference
I use search as well via openwebui + searxng.