Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local Qwen isn't a worse Opus, it's a different tool

blog.alexellis.io

Post reply on HN
Print view View original

141–150 of 271 posts

Re: Local Qwen isn't a worse Opus, it's a different tool

#141

Post by Wowfunhappy »

glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

> With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative. Also (you might raise an eyebrow at this) being nice to Claude will be rewarded and being mean to Claude will be punished. Claude tends to mirror your tone more aggressively and you don't want to get into negative loops with it.

> With GPT, you have to be precise and reduce ambiguity. GPT will often try to resolve ambiguity in a min-max style "I'm going to do X, but make sure it is not quite Y". It will tend to be more paranoid and overengineer to catch all edge cases if you don't tell it precisely what the scope is.

I agree with all of this except for one thing: I swear to god, being mean to Claude at the right time can be enormously effective. The F-bomb in particular seems to really help it snap out of ruts sometimes.

Re: Local Qwen isn't a worse Opus, it's a different tool

#143

Post by Wowfunhappy »

cassianoleal wrote: post #125
coldtea wrote: post #103

Earlier quoted context omitted.

> I would love to have a product sheet showing what each models strengths an weaknesses are, so that I can have a clear decision tree of "if this kind of work, use model X", or "model Y should be used in ways Z". But they all look the same from the outside and the only way to figure out which might be marginally better at what is to do extensive, time consuming, and perhaps expensive testing. Think of it less like a…

They are not human. Humans have names, faces, voices, personality, a personal history, family, care for whatever they call their community. With humans it's actually good and worthwhile to create and strengthen connections. With an LLM, that's psychosis.

They're not human. But they are trained on human language, and thinking of them as similar to a human helps me work with them effectively.

Re: Local Qwen isn't a worse Opus, it's a different tool

#144

Post by epolanski »

glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

[dead]

Re: Local Qwen isn't a worse Opus, it's a different tool

#145

Post by epolanski »

dkersten wrote: post #80
glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

> This is not scientific at all, just vibes, YMMV. This is the problem. I would love to have a product sheet showing what each models strengths an weaknesses are, so that I can have a clear decision tree of "if this kind of work, use model X" , or "model Y should be used in ways Z" . But they all look the same from the outside and the only way to figure out which might be marginally better at what is to do extensive,…

The problem is that this is very hard to replicate and benchmarks focus on E2E tests, going from one prompt to the final solution.

They do not test how models perform when used interactively, like most of us do.

Re: Local Qwen isn't a worse Opus, it's a different tool

#146

Post by hypfer »

sleepyeldrazi wrote: post #136
sanderjd wrote: post #15

Earlier quoted context omitted.

Right. Opus 4.5 8 months ago, good enough for agentic coding. How far behind that are open weight models? More than 8 months? But how much more? When will they reach Opus 4.5 level? A few months from now? A year from now? Never?

Opus also has a deeply ingrained personality that always de-rails sneakily into what it's taught, not what the user intends. This is good if the user doesn't know the details of the work they need performed and a huge time waste when the user knows exactly how something needs to be implemented. I have found claude models, especially fable, to be impossible to work with when the work requires reading papers from days…

> I have multiple long sessions with opus (not as many with fable as it got taken down quickly) where it keeps fighting me on problems, sayings "that's not how it works" / "that is not possible", followed by me linking the paper (after i've told it to actually read up on the latest research in this field), and it hits me with the usual "You were right.".

I genuinely do not understand why people not only just put up with this but also pay _a lot of money_ for the _privilege_ of doing so.

It's like having _the worst_ colleague but you actually go out of your way to talk with the guy. Why.

Re: Local Qwen isn't a worse Opus, it's a different tool

#147

Post by user43928 »

madeofpalk wrote: post #110
coldtea wrote: post #103

Earlier quoted context omitted.

> I would love to have a product sheet showing what each models strengths an weaknesses are, so that I can have a clear decision tree of "if this kind of work, use model X", or "model Y should be used in ways Z". But they all look the same from the outside and the only way to figure out which might be marginally better at what is to do extensive, time consuming, and perhaps expensive testing. Think of it less like a…

Except, where every different model and version is like a different person where you need to learn their idiosyncrasies of how they work every other month. It's a very very bizarre way to use a computer. Personally, I just don't. I'll use and prompt the LLMs the way that feels natural to me and move on with my life. Maybe I don't always get completely optimal results from them, but im also not spending half my day pl…

I also don't think I need to prompt Claude differently than Codex.

The most important thing to be aware of in my opinion would be that Claude is better at UI design, and leaves a lot more comments in the code.

Other than that the results seem similar, at least functionally. I do not usually review the code style.

Re: Local Qwen isn't a worse Opus, it's a different tool

#148

Post by hypfer »

eurekin wrote: post #129
trey-jones wrote: post #109

Earlier quoted context omitted.

I'm really curious about this, not because I disagree, but because I want to avoid agents going whack. Are you running vllm for yourself only, or a for a team, or for an application, etc? And do you feel there is a minimum hardware requirement for vllm to be useful in this way? My weekend project is going to be building a home inference server (from ancient datacenter parts) and I'm still massaging in my head what th…

If I started today, with building a server, I'd jump right into verified set-ups and writeups, like this one: https://github.com/noonghunna/club-3090 You can find info about running a patched version of vllm for 1x24gb, 2x and 4x. There's also quite a few "blackwell" subreddits, where people seem to share a lot of substantial information, if you're going the 6000 route.

That writeup is completely unhinged and utterly incomprehensible to follow.

It just throws "you can do " at you, with no real explainer regarding how it manages that and which trade-offs are made. I still don't know for certain, but I think one of those trade-offs is 3 bit context? Which is a terrible idea.

Please don't share these walls of noise. They shouldn't exist

Re: Local Qwen isn't a worse Opus, it's a different tool

#149

Post by user43928 »

cube00 wrote: post #58
h05sz487b wrote: post #45

Earlier quoted context omitted.

> It is very much like playing an instrument. Or it is more like playing a slot machine and you imagine the rest.

This is how I feel whenever I see bold all caps instructions in a system prompt or someone claims they conducted "research" and found the magic prompt template that makes the model pay out. Maybe it works some of the time but it isn't a solution that works everytime. It reminds me of people hovering to play a slot machine when someone gets up and it hasn't paid out as if they've solved slot machines. While I don't mi…

Has there been any evidence of a well known provider rerouting to lower quality models?

Last I saw, engineers working at OpenAI denied this on HN.

I saw that someone set up a tracker that aims to record the performance of the models, and so far it has not shown any statistically significant deviation in performance for Codex, and not yet enough data for Claude: https://marginlab.ai/trackers/codex/

Re: Local Qwen isn't a worse Opus, it's a different tool

#150

Post by BoorishBears »

krzyk wrote: post #113
theplumber wrote: post #19

Earlier quoted context omitted.

I think in the next 6 months we will have Opus 4.5 performance in open models. We are very close

We need first to reach level of Sonnet 4.x, we aren't at that level yet.

GLM 5.2 is comfortably at Sonnet 4 at the very least. Same with Minimax M3
Post reply on HN