Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local Qwen isn't a worse Opus, it's a different tool

blog.alexellis.io

Post reply on HN
Print view View original

241–250 of 271 posts

Re: Local Qwen isn't a worse Opus, it's a different tool

#241

Post by killix »

weitendorf wrote: post #77
tym0 wrote: post #69

Earlier quoted context omitted.

I feel like this is really due to the harness. Gemini CLI at work has the same issue: it'll prefer hacking your workstation over just asking you how to proceed. I think the harnesses are setup to have a bias to action otherwise the LLM would just stop all the time when doing trivial task but it also mean they'll keep going when the "obvious" path is to just prompt the user.

While I agree that the harness is part of it, I think it's also a lack of epistemic understanding or awareness for what it means to actually solve a problem vs just get something kinda working; maybe if Claude Code or other harnesses made web search more likely or had a better way to make technical documentation and specs available to models, it would be better solvable there. I often tell it to stop asking me and ju…

[flagged]

Re: Local Qwen isn't a worse Opus, it's a different tool

#242

Post by Terretta »

throwa356262 wrote: post #95
theshrike79 wrote: post #43

Earlier quoted context omitted.

The power of Opus isn't just the model, it's in the harness too. You can try it by using Opus through Github Copilot vs official Anthropic tools. You'll get very different results and experience (in my opinion).

open source harnesses are also improving rapidly. Some people would claim they are already far better than CC and Codex.

which harnesses, and which when?

Re: Local Qwen isn't a worse Opus, it's a different tool

#243

Post by mekdoonggi »

chadgpt3 wrote: post #181
skipants wrote: post #179

I feel like it's the Emperor's new clothes reading this article and seeing the praise it's getting. This sentence doesn't even make sense: > These products use very low level Linux primitives like containers, Kubernetes, Firecracker microVMs, and networked protocols. Out of anything that is a "low level linux primitive" I could maybe argue that networking? protocols fit the bill. And it's obviously fully AI-generated…

Low level today means JavaScript instead of typescript

Low-level today means opening IDE instead of the Chat client.

Re: Local Qwen isn't a worse Opus, it's a different tool

#244

Post by d0mine »

Terretta wrote: post #242
throwa356262 wrote: post #95

Earlier quoted context omitted.

open source harnesses are also improving rapidly. Some people would claim they are already far better than CC and Codex.

which harnesses, and which when?

Mimocode(opencode), Pi Coding Agent

Re: Local Qwen isn't a worse Opus, it's a different tool

#245

Post by fittingopposite »

3abiton wrote: post #70
zmmmmm wrote: post #9

That's a great write up. The one thing I feel it seems to under estimate is the likelihood of improvement. Even the authors acknowledge it's not even worth comparing local models from a year ago to what we have now. In fact, people widely see Opus 4.5 in November last year - 8 months ago - as the first time agentic coding became viable broadly viable even with frontier hosted models. So why would we lock in hard on a…

And a big thing that's missing is ... the harness comparison. Ot plays a very big role. I use forge, and I have been inpressed with what it can do given all the limitations of local models.

How would you benchmark them? Are there any benchmarks for harnesses?

Re: Local Qwen isn't a worse Opus, it's a different tool

#246

Post by cptskippy »

hbbio wrote: post #11
cptskippy wrote: post #4

I've been running qwen3-5-9b-q4-k-m and qwen3-6-27b-q6-k simultaneously on an Intel Arc Pro B70 with a lot of success. https://github.com/cptskippy/battlemage-llm-gateway Opencode has been a huge productivity accelerator. I have two Hermes agents that I'm training to support my workflow with pretty good success. One is a personal assistant who manages my backlog and keeps me on task, follows up with me on items, and…

Interesting setup, thx for sharing. How many tokens/sec do you get with 27b? Are you using MTP?

I haven't done any in-depth synthetic benchmarks but I had my Hermes agent run some and I ran a couple directly on the LLM Gateway that showed similar results.

Hermes reported 18.45 tok/s consuming the llama-swap endpoint across the wire. Locally I got 19-19.1 tok/s on the gateway. I'm running the Qwen 3.6 27B Q6 model (qwen3-6-27b-q6-k) off LM Studio and it's less than 0.3s to first token.

It's not good for conversational use cases as it can take 1-2 minutes to respond to a prompt.

I have two Hermes Profiles running, one is a personal assistant that manages my backlog and provides me morning reminders, solicits for evening updates, and will run overnight research projects for me. The other profile is a coding helper for personal projects. I can ask it to make changes and it will churn for 15 minutes, submit a PR, and notify me that the PR is ready to review. It's faster than me at basic coding tasks.

Re: Local Qwen isn't a worse Opus, it's a different tool

#247

Post by roxolotl »

CamperBob2 wrote: post #223
alexellisuk wrote: post #211

Earlier quoted context omitted.

Fair enough, that sentence was fairly compressed. I’ve reworded it - the meaning remains the same. The post is not AI generated, I use AI for code generation and write my own articles. Which part of the post are you struggling with? This is a post describing our own experience and journey. Happy to back up any specific claim.

How about your reply here? Was that AI-generated? If not, are you conscious of how much you're starting to sound like AI? Is that something you see as a positive thing, or something you'd like to avoid? I actually find this somewhat interesting, because it seems that a lot of people who weren't comfortable with expressing themselves verbally are feeling more empowered in that area. We're hearing new voices for the fi…

Interestingly I haven’t had such a challenge in the past but regularly am accused of being AI. In my case I think it’s because I have experience writing philosophy papers which trend towards a more stilted tone. For this response I’m also probably exaggerating it a tad.

A lot of it is that good argumentative/persuasive writing follows the structures that AI writing follow. Groups of three, not only x but y, etc. It’s all stuff that’s considered a best practice. It used to be comments structured like that meant you were writing overly formally now it’s so common people see it as LLM output.

Oh as for how I feel about this… couldn’t care less if people think I’m AI or not. Im not in it for the karma, I don’t care if people think I use AI even as someone who’s anti AI.

This comment of mine has discussion of this topic https://news.ycombinator.com/item?id=48035093

Re: Local Qwen isn't a worse Opus, it's a different tool

#248

Post by boomskats »

hypfer wrote: post #146
sleepyeldrazi wrote: post #136

Earlier quoted context omitted.

Opus also has a deeply ingrained personality that always de-rails sneakily into what it's taught, not what the user intends. This is good if the user doesn't know the details of the work they need performed and a huge time waste when the user knows exactly how something needs to be implemented. I have found claude models, especially fable, to be impossible to work with when the work requires reading papers from days…

> I have multiple long sessions with opus (not as many with fable as it got taken down quickly) where it keeps fighting me on problems, sayings "that's not how it works" / "that is not possible", followed by me linking the paper (after i've told it to actually read up on the latest research in this field), and it hits me with the usual "You were right.". I genuinely do not understand why people not only just put up w…

I agree 100%. All of the models do it to some extent after the context gets tired, but opus is the worst and the sneakiest. And even when you do coerce it into doing what you want it feels like something out of r/maliciouscompliance. Much more so than most non-anthropic models. Way more so than codex/gpt or even gemini.

also thanks for my l10spuh :)

Re: Local Qwen isn't a worse Opus, it's a different tool

#249

Post by hashmap »

notduncansmith wrote: post #65
hashmap wrote: post #30

Earlier quoted context omitted.

totally true. one key for claude is to not smell like an evaluator, its good at knowing when its being tested and will behave defensively and avoid doing work. i avoid this basin by typing unreasonably excited about the thing i want done. like way over the top. it's harder to keep that up than it sounds.

I’m able to avoid this basin with a pretty natural baseline professional positivity and frustration management that I would employ with pair-programming. For example, if I just made progress with a human I was guiding through a task, I would be like “Nice, now let’s xyz” (instead of just “now let’s xyz” as if _I_ were the robot lol) or if we had to work for a result I’ll be like “Sweet! Looks good, now let’s xyz” - t…

definitely a lot of the same reframing of a result that would work well with people works well with agents too, definitely around the emotional regulation aspect. frustration just goes bad places if you linger there. though, i get the best results when just ditching the professionalism entirely and talk like i text. the professional voice is a really narrow bottleneck to project signal through and lets things be underdetermined when they dont need to be, or at least thats how it's worked out for me

Re: Local Qwen isn't a worse Opus, it's a different tool

#250

Post by hashmap »

glerk wrote: post #39
hashmap wrote: post #30

Earlier quoted context omitted.

totally true. one key for claude is to not smell like an evaluator, its good at knowing when its being tested and will behave defensively and avoid doing work. i avoid this basin by typing unreasonably excited about the thing i want done. like way over the top. it's harder to keep that up than it sounds.

at the risk of sharing my secret magic spells :) > this is phenomenal work, genuinely! I feel like you read my mind! can go a long way. of course, I would only say that when I mean it, because Claude can get superficial and cut corners which is why I prefer GPT for raw implementation.

def like having a couple packets of copypasta shortcuts the emotional labor lol. it reliably works because every new session the agent has forgotten you ever existed
Post reply on HN