Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local Qwen isn't a worse Opus, it's a different tool

blog.alexellis.io

Post reply on HN
Print view View original

151–160 of 271 posts

Re: Local Qwen isn't a worse Opus, it's a different tool

#151

Post by nosyke »

glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

It's interesting because this really hasn't been my experience over the last month or two. I would prior it was, but it's definitely changed on my end. In my exp I've needed to be way more specific with Claude and with Codex I can generally approach a problem in a much more open ended way.

Re: Local Qwen isn't a worse Opus, it's a different tool

#152

Post by mystraline »

> We've all heard people say that local Qwen 27B or 35-A3B is "near-Opus level"

Uh, so, yeah. Im running local Qwen, but Qwen3.5-122B using Krasis https://github.com/brontoguana/krasis

Its by far better than Opus.

In fact with a phone migration, I was using an OLD android 2fa app "andOTP". Backup files it emitted were JSON but not any sort of standard.

I needed the standards version using otpauth:// to upload in my current 2fa. And gave it to my local qwen3.5-122b.

It responded with a scary "you uploaded credentials to a public instance LLM! And, it emitted standards compliant URLs. The new app "Tokn" ingested just fine. When side by side was tested, everything was 100% correct.

I coukd have did it myself, but it was a one-off. And asking local Qwen worked perfectly. Took like 6 minutes. Would have taken me 1h.

Re: Local Qwen isn't a worse Opus, it's a different tool

#153

Post by andai »

glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

I asked GLM 5.2 for a HTML5 port of my old C#/XNA game. It ported all the code exactly (except for operator overloading, which doesn't exist in JS), and added more code to make the code work.

I asked Claude Sonnet 4.6 for the same thing, and Claude's version was like if the game had been written in JS originally.

Also, for some reason it made it a single HTML file, removed all assets, dynamically generated graphics and dynamically generated music. It also gave me a new, better background.

This surprised me, since it was not what I asked for. I just asked it to port the game.

I was pretty pleased about the choices it made, but I'm not sure how to turn that behavior on and off. Sometimes you want it to be creative, sometimes you want it to actually do what you said.

Re: Local Qwen isn't a worse Opus, it's a different tool

#154

Post by Forgeties79 »

tekne wrote: post #128
palata wrote: post #96

Earlier quoted context omitted.

While I agree with you, I think it's diverging from the initial point. The original take was "LLMs are very much like playing an instrument". I think they are very much NOT like playing an instrument. While different musicians will produce different results, one musician won't get drastically different results on different days or when trying a different "copy" of the same instrument. If you can play the violin on yo…

Will you? That's only if both violins are tuned the same way, and one must continually tune them lest they get out of sync. Similarly, an LLM can be extremely consistent if tuned properly -- indeed, if you fix the weights and settings, they can be made "essentially deterministic" for many prompts!

Anyone who has even modest experience with a particular instrument can pick any one up at any time and play it. The way the notes are played is consistent and produces a consistent note. If you tune 50 guitars to standard, the chords all produce what they should., It is a predictable instrument. You do not pick up a trumpet in one place then another and find the key combinations are suddenly different.

You know what we are talking about. Tuning, poor playing, all of that is mild variation from what we know it is supposed to do every time and we can target the the notes they are supposed to hit consistently. You're comparing slight tonal variations to completely different outputs from the same inputs. If I hit a "C" on the piano, it is going to play "C." If it does not, then the piano is not functioning properly. LLM's for some reason get a pass on this and it makes them very distinct from musical instruments.

This feels like a very nitpicky steel man, not a productive attempt at discussion.

Re: Local Qwen isn't a worse Opus, it's a different tool

#155

Post by itsthecourier »

wanted sovereignty, bought a Blackwell for usd12k, discovered a billing issue in some customer and explains that will cover the card

I don't follow how it supports the decision of buying the card, I would even say using online SOTA models would had caught it earlier without usd12k and monthly electricity being spent

Re: Local Qwen isn't a worse Opus, it's a different tool

#156

Post by user43928 »

theshrike79 wrote: post #43
sanderjd wrote: post #15

Earlier quoted context omitted.

Right. Opus 4.5 8 months ago, good enough for agentic coding. How far behind that are open weight models? More than 8 months? But how much more? When will they reach Opus 4.5 level? A few months from now? A year from now? Never?

The power of Opus isn't just the model, it's in the harness too. You can try it by using Opus through Github Copilot vs official Anthropic tools. You'll get very different results and experience (in my opinion).

GitHub Copilot in vscode has two ways to access Opus: the Copilot harness or the Claude Code Agent SDK within Copilot.

And that's if we assume that the vscode GHCP default Agent ("Local") is the same as the "Copilot CLI" one that is also selectable in vscode. I have not tried that one.

A few weeks ago the Claude Code Agent SDK was much better than the default Copilot Agent, but nowadays I am not sure.

Re: Local Qwen isn't a worse Opus, it's a different tool

#157

Post by renegade-otter »

ACCount37 wrote: post #111
coldtea wrote: post #103

Earlier quoted context omitted.

> I would love to have a product sheet showing what each models strengths an weaknesses are, so that I can have a clear decision tree of "if this kind of work, use model X", or "model Y should be used in ways Z". But they all look the same from the outside and the only way to figure out which might be marginally better at what is to do extensive, time consuming, and perhaps expensive testing. Think of it less like a…

One issue with that is that human helpers last longer. LLMs cycle in and out in months, and what held for Your Favorite LLM 6.7 may not hold for Your Favorite LLM 6.9.

Right, this is why I would slam the breaks on investing into your workflow all of your time and effort, because 2 months from now it may be out the window. Frontier models are also constantly being tweaked, so what worked yesterday may be off today.

ChatGPT was obedient with the grill-me technique, just wrote a plan. Yesterday it started jumping to implementation. Why?

Re: Local Qwen isn't a worse Opus, it's a different tool

#158

Post by alexellisuk »

itsthecourier wrote: post #155

wanted sovereignty, bought a Blackwell for usd12k, discovered a billing issue in some customer and explains that will cover the card I don't follow how it supports the decision of buying the card, I would even say using online SOTA models would had caught it earlier without usd12k and monthly electricity being spent

Author here. Thanks for the question. I'll answer assuming this is a question you have for me.

As explained in the post - the 3090s were what were the test bed that proved the investment was worth it. Customer support, architecture reviews, telemetry to check license compliance. None of that could be done with online models. The amount of time we can spend going backwards and forth with enterprise customers over email can really amplify costs to our team. A few actual issues we found and fixed were listed on the linked blog post: https://www.openfaas.com/blog/painless-support-with-diag/

Having recovered revenue using it in an airgap, to preserve data agreements was more of a cherry on the cake. No need to worry about the investment, it's covered itself.

Hope that helps.

Re: Local Qwen isn't a worse Opus, it's a different tool

#159

Post by Planktonne »

coldtea wrote: post #105
cube00 wrote: post #58

Earlier quoted context omitted.

This is how I feel whenever I see bold all caps instructions in a system prompt or someone claims they conducted "research" and found the magic prompt template that makes the model pay out. Maybe it works some of the time but it isn't a solution that works everytime. It reminds me of people hovering to play a slot machine when someone gets up and it hasn't paid out as if they've solved slot machines. While I don't mi…

> This is how I feel whenever I see bold all caps instructions in a system prompt or someone claims they conducted "research" and found the magic prompt template that makes the model pay out. Maybe it works some of the time but it isn't a solution that works everytime. For such thing to be useful, it's enough that they works substantially more times that not having those instructions in.

Every gambler thinks their system works, given enough chances.

Re: Local Qwen isn't a worse Opus, it's a different tool

#160

Post by alexellisuk »

whazor wrote: post #57

Would be interesting to use local models for: - tool calling - code base exploration - anonymizing / abstracting your request Such that your local AI communicates to frontier model like an expensive consultant giving high level advice. I think due to the lower latency of a local model that this could be faster.

One of the things I mentioned in the post:

> Local models can quickly read and explain codebases, even if they can't write them - this is a superpower

Might have been buried lower down.

And yes latency of local on a fast card with MTP enabled can be blistering 130-200 tokens per second sustained at full context on Q5. About 100+ on Q8.

On tool calling

> Agent Skills can help immensely - we had a local agent set up Slicer completely from scratch on a new mini PC. It even gave feedback on the usability of slicer CLI which we integrated

There's a link to a post showing some examples.

Occasionally, we'll also have the local model _review_ the changes of GPT/Opus - and it can return duds, but also insights the larger model overlooked, or was too intelligent to pick out.

So yes - absolutely blazing fast at understanding a codebase, very good at running skills "cheaply" and could be used with larger models as a "helper" / sub-agent.

Post reply on HN