Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local Qwen isn't a worse Opus, it's a different tool

blog.alexellis.io

Post reply on HN
Print view View original

261–270 of 271 posts

Re: Local Qwen isn't a worse Opus, it's a different tool

#261

Post by nrb »

redrove wrote: post #68
theshrike79 wrote: post #46

Earlier quoted context omitted.

My dream would be a local model that can do, say, 80% of the day to day tasks I need; "how does X Handler connect to Y storage?", "commit that feature, but leave out the bits that relate to billing" etc. It would have 99% reliable tool calling - and most importantly - the ability to go "this task is beyond my skills" and refer to a Big Boy Online Model in a gigantic datacenter somewhere. This way all of the simple st…

> My dream would be a local model that can do, say, 80% of the day to day tasks I need; "how does X Handler connect to Y storage?", "commit that feature, but leave out the bits that relate to billing" etc. Qwen 3.6 27B can do that today, but setup properly and in a good quant, I run an autoround [0] with weights in int8 and attention heads in f16 on a single RTX 6000 Pro Blackwell Max-Q via vllm with mtp=2 and full c…

[dead]

Re: Local Qwen isn't a worse Opus, it's a different tool

#262

Post by probiz »

redrove wrote: post #68
theshrike79 wrote: post #46

Earlier quoted context omitted.

My dream would be a local model that can do, say, 80% of the day to day tasks I need; "how does X Handler connect to Y storage?", "commit that feature, but leave out the bits that relate to billing" etc. It would have 99% reliable tool calling - and most importantly - the ability to go "this task is beyond my skills" and refer to a Big Boy Online Model in a gigantic datacenter somewhere. This way all of the simple st…

> My dream would be a local model that can do, say, 80% of the day to day tasks I need; "how does X Handler connect to Y storage?", "commit that feature, but leave out the bits that relate to billing" etc. Qwen 3.6 27B can do that today, but setup properly and in a good quant, I run an autoround [0] with weights in int8 and attention heads in f16 on a single RTX 6000 Pro Blackwell Max-Q via vllm with mtp=2 and full c…

[flagged]

Re: Local Qwen isn't a worse Opus, it's a different tool

#263

Post by klardotsh »

d0mine wrote: post #244
Terretta wrote: post #242

Earlier quoted context omitted.

which harnesses, and which when?

Mimocode(opencode), Pi Coding Agent

OpenCode is a quirky, buggy mess in my experience, but Pi is pretty solid and I’m glad to have recently switched over. Zerostack looks promising for certain types of users too.

Re: Local Qwen isn't a worse Opus, it's a different tool

#265

Post by throwa356262 »

klardotsh wrote: post #263
d0mine wrote: post #244

Earlier quoted context omitted.

Mimocode(opencode), Pi Coding Agent

OpenCode is a quirky, buggy mess in my experience, but Pi is pretty solid and I’m glad to have recently switched over. Zerostack looks promising for certain types of users too.

Mimocode from Xiaomi is a fork of opencode with some improvements.

There is also reasonix, which specializes in good caching tlto keep costs low;

https://reasonix.io/

I agree opencode can be a bit buggy, but codex and claude code are not much better either.

Re: Local Qwen isn't a worse Opus, it's a different tool

#266

Post by user43928 »

dannyw wrote: post #260
user43928 wrote: post #149

Earlier quoted context omitted.

Has there been any evidence of a well known provider rerouting to lower quality models? Last I saw, engineers working at OpenAI denied this on HN. I saw that someone set up a tracker that aims to record the performance of the models, and so far it has not shown any statistically significant deviation in performance for Codex, and not yet enough data for Claude: https://marginlab.ai/trackers/codex/

Yes, OpenAI admits they silently reroute sensitive requests to different models for user welfare at least: https://openai.com/index/building-more-helpful-chatgpt-exper... The implementation was so borked, SamA went back on Reddit and apologised: https://old.reddit.com/r/ChatGPT/comments/1o6jins/updates_fo... Model re-routing happens for coding tasks too. For example, in OpenAI support pages used to (at least 1 month…

What you linked appears to be related to GPT-5's Auto router in the ChatGPT app back then, and that it supposedly would choose the 'good' model over the pretty bad Instant model for mental health requests.

That's pretty far from the hypothesis that either OpenAI or Anthropic is using adaptive quantisation based on load for their professional coding agent tools.

This is what I think engineers working at OpenAI explicitly denied, and for which we have seen zero evidence yet.

Many people seem to believe it anyway, but the non-deterministic nature of the tools appears to be the more plausible explanation for perceived degradation, in my opinion.

Re: Local Qwen isn't a worse Opus, it's a different tool

#267

Post by Stitch4223 »

mekdoonggi wrote: post #243
chadgpt3 wrote: post #181

Earlier quoted context omitted.

Low level today means JavaScript instead of typescript

Low-level today means opening IDE instead of the Chat client.

That was 16 hours ago. Mind captioning to my smartphone and seeing solutions appear without uttering a word or lifting a finger.

Re: Local Qwen isn't a worse Opus, it's a different tool

#268

Post by demades »

dkersten wrote: post #80
glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

> This is not scientific at all, just vibes, YMMV. This is the problem. I would love to have a product sheet showing what each models strengths an weaknesses are, so that I can have a clear decision tree of "if this kind of work, use model X" , or "model Y should be used in ways Z" . But they all look the same from the outside and the only way to figure out which might be marginally better at what is to do extensive,…

[flagged]

Re: Local Qwen isn't a worse Opus, it's a different tool

#269

Post by te0006 »

mystraline wrote: post #152

> We've all heard people say that local Qwen 27B or 35-A3B is "near-Opus level" Uh, so, yeah. Im running local Qwen, but Qwen3.5-122B using Krasis https://github.com/brontoguana/krasis Its by far better than Opus. In fact with a phone migration, I was using an OLD android 2fa app "andOTP". Backup files it emitted were JSON but not any sort of standard. I needed the standards version using otpauth:// to upload in my c…

Interesting setup. What GPU(s)/VRAM, CPU and RAM are you using for the 122B model, with which quantization, and what token rates do you achieve for prefill and generation?

Re: Local Qwen isn't a worse Opus, it's a different tool

#270

Post by milch »

Wowfunhappy wrote: post #141
glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

> With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative. Also (you might raise an eyebrow at this) being nice to Claude will be rewarded and being mean to Claude will be punished. Claude tends to mirror your tone more aggressively and you don't want to get into negative loops with it. > With GPT, you have to be precise and…

I'm never mean but sometimes when Claude does something especially boneheaded I just hit it with a single "bruh". That usually triggers an automatic "You're absolutely right -- I shouldn't have X and followed your directions more closely, let me revert and do Y instead"
Post reply on HN