Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local Qwen isn't a worse Opus, it's a different tool

blog.alexellis.io

Post reply on HN
Print view View original

201–210 of 271 posts

Re: Local Qwen isn't a worse Opus, it's a different tool

#201

Post by scotty79 »

chadgpt3 wrote: post #182
scotty79 wrote: post #161

Earlier quoted context omitted.

> Use of some tools like LLMs might be more inducing psychosis than others like plain compilers or hammers. I really don't get it. Why the fact that it outputs words is so goddamn important for everybody? How does it suddenly make you so emotionally vulnerable? Does my brain work in a different way than the rest of humanity? Can't you disregard what's irrelevant? Is every programmer suddenly a trump supporter that ha…

It's a certain percentage of people and yes it's different for them because it outputs words and triggers some kind of emotional trust response.

As good opportunity as any to acquire some emotional intelligence.

Re: Local Qwen isn't a worse Opus, it's a different tool

#202

Post by Wowfunhappy »

mcbits wrote: post #193
Wowfunhappy wrote: post #141

Earlier quoted context omitted.

> With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative. Also (you might raise an eyebrow at this) being nice to Claude will be rewarded and being mean to Claude will be punished. Claude tends to mirror your tone more aggressively and you don't want to get into negative loops with it. > With GPT, you have to be precise and…

I haven't really experimented with being "nice" or "mean", but I would worry that a prompt like "No, dumbass, ..." would kick it into the patterns of someone who frequently got called a dumbass (perhaps for good reason) in the training set. On the other hand, maybe it could trigger more defensive responses with argumentation to explain its conclusions.

I only use it for behaviors I really want the model to clamp down on, and I don't think I've ever told the model it was stupid. But I might say something like:

    No, don't f***ing do that! What part of "[previous instruction]" don't you f***ing understand? I am extremely angry and disappointed by your inability to [whatever]. Do better please.
> maybe it could trigger more defensive responses with argumentation to explain its conclusions.

Quite the opposite, it makes the model extremely conciliatory—which in this situation is what I want. If you're hoping to make the model less sycophantic, this is the wrong tool.

Re: Local Qwen isn't a worse Opus, it's a different tool

#203

Post by duckmysick »

skydhash wrote: post #197
duckmysick wrote: post #172

Earlier quoted context omitted.

I'd argue the opposite. Technology in the past few decades was (is) limited and humans had to adapt to it. We communicate with other humans using voice and three dimensional hand gestures. To use computers and early phones we had to learn to operate new input devices: keyboards and mice. Later with touchscreens we moved to two dimensional hand (finger) gestures. We're barely making voice commands work with our device…

> Technology in the past few decades was (is) limited and humans had to adapt to it. Was it? Think first about what it replaced. Lots of manual computation in bookkeeping and financial sectors. Telegrams and snail mail moved to email. Typesetting in books and magazines became easier and widely available,… If there’s one thing that you can’t say about computers is that they’re limited.

No doubt that computers enabled a lot of automation. We can both agree with that.

The context was that technology should evolve to fit the humans [not the other way around]. And if contemporary technology didn't have limitations, it would be correct.

But it did and humans had to adapt to the computers. Humans had to develop and learn special languages so they could communicate with computers to do all those useful things you mentioned. Why? They were limited in understanding (or parsing) human languages. It took us decades before we could talk to computers in human languages. We're getting pretty close - especially in the past few years - but there's still some friction.

Re: Local Qwen isn't a worse Opus, it's a different tool

#204

Post by cube00 »

user43928 wrote: post #149
cube00 wrote: post #58

Earlier quoted context omitted.

This is how I feel whenever I see bold all caps instructions in a system prompt or someone claims they conducted "research" and found the magic prompt template that makes the model pay out. Maybe it works some of the time but it isn't a solution that works everytime. It reminds me of people hovering to play a slot machine when someone gets up and it hasn't paid out as if they've solved slot machines. While I don't mi…

Has there been any evidence of a well known provider rerouting to lower quality models? Last I saw, engineers working at OpenAI denied this on HN. I saw that someone set up a tracker that aims to record the performance of the models, and so far it has not shown any statistically significant deviation in performance for Codex, and not yet enough data for Claude: https://marginlab.ai/trackers/codex/

> Has there been any evidence of a well known provider rerouting to lower quality models?

The firm [Anthropic] would deliberately degrade the model’s performance in ways that were invisible to the user.

https://news.ycombinator.com/item?id=48485958

Re: Local Qwen isn't a worse Opus, it's a different tool

#205

Post by zahlman »

glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

> being nice to Claude will be rewarded and being mean to Claude will be punished

... That does sound like something that Anthropic would deliberately aim for, yeah.

> With GPT, you have to be precise and reduce ambiguity.

I have found that it occasionally makes a wild misinterpretation, that makes a bit of sense in retrospect given how I worded something but is still surprising.

It also sometimes tries to loop in and tie together ideas from earlier in the conversation that really shouldn't still appear relevant. But that might be a general LLM thing.

Re: Local Qwen isn't a worse Opus, it's a different tool

#206

Post by keeganpoppen »

glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

this is the best distillation of what various models are like that i've ever heard... it's wild to me that people view LLMs as this monolithic entity, like "how do i get the best prompts to do ?", when it is such a clearly interactive medium, but the returns to engaging with the various models and understanding their "vibes" are very, very high.

Re: Local Qwen isn't a worse Opus, it's a different tool

#207

Post by ttsiodras »

Interesting article.

IMHO, the author could have done two things better:

- vllm instead of llama.cpp. With NVIDIA HW, there is huge difference in multi-user loads and caching with vllm; when he was complaining about what happens when more than one user uses the model, and about losing caching, I was "well, duh".

- The budget he used for a single card could have instead be put to far, far better use with SPARKs. I have access to a cluster of 2 x GX10 - total cost less than half what he paid, even today - and I am running vllm and Deepseek v4 Flash. The difference compared to any Qwen is tremendous - I've NEVER seen it loop, and in all my experiments so far, it's the most Sonnet-y model I've ever tried (antirez seems to agree, hence his ds4 fork).

If you're wondering about how I set it up in the 2 GX10s: https://forums.developer.nvidia.com/t/deepseek-v4-flash-offi...

Performance: 2K t/s prefill ( very useful for feeding tons of source code into its massive context window ) and around 50-60 tg/s in my coding sessions in the pi.dev harness. With the money the author paid, he could have bought 4 GX10s, and double both numbers ( vllm basically scales almost linearly with tensor parallelism ).

Re: Local Qwen isn't a worse Opus, it's a different tool

#208

Post by furyofantares »

glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

I strive to make this NOT the case, by fixing up my skills or agents.md whenever they don't work how I want in one provider or the other. I mean, yeah, it would be awesome if I was a virtuoso with all the agents/models I use. But I am switching all the time, either because one leapfrogs the other, or because I hit limits (I'm on $200/mo on both Claude and Codex, and also subscribe to some others when I hit limits on both of those simultaneously).

Re: Local Qwen isn't a worse Opus, it's a different tool

#209

Post by zahlman »

theshrike79 wrote: post #36
glerk wrote: post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

Yyep. IME Claude is the most "creative" of the bunch, you can get surprising ideas out of it that were kinda tickling the back of your head but didn't really connect. BUT it's also "relentlessly proactive" like simonw put it. It _will_ get the job done, it's the smartest idiot in town. Why use a library to parse $format when you can just write a custom 1000 line parser? Or if it can't access something, it'll pursue t…

FWIW I find that GPT can be very creative when discussing a high-level design. Once it starts writing code snippets it will offer to take things in a bunch of different directions.

Re: Local Qwen isn't a worse Opus, it's a different tool

#210

Post by skydhash »

duckmysick wrote: post #203
skydhash wrote: post #197

Earlier quoted context omitted.

> Technology in the past few decades was (is) limited and humans had to adapt to it. Was it? Think first about what it replaced. Lots of manual computation in bookkeeping and financial sectors. Telegrams and snail mail moved to email. Typesetting in books and magazines became easier and widely available,… If there’s one thing that you can’t say about computers is that they’re limited.

No doubt that computers enabled a lot of automation. We can both agree with that. The context was that technology should evolve to fit the humans [not the other way around]. And if contemporary technology didn't have limitations, it would be correct. But it did and humans had to adapt to the computers. Humans had to develop and learn special languages so they could communicate with computers to do all those useful th…

> Humans had to develop and learn special languages so they could communicate with computers to do all those useful things you mentioned. Why? They were limited in understanding (or parsing) human languages

You may need to revisit your computation theory courses. Computers are the embodiment of a mathematical model and thus the inputs and outputs are formalized.

Do you just hold a pen and words are written automatically? Do you just hover your hands over a piano and have the moonlight sonata played? No, you have to do precise mechanical movements because that’s how the output is realized.

There’s no such things as words, sentences, keywords, statements at the computer level. What it does is symbol manipulation. You provide it a string of symbols, the rules for the manipulation, and it will provide a string of symbols as the output.

What symbols, what rules, are completely arbitrary . We just found that {1,0} are all that we needed as the set of symbols and that Context-Free Grammar is perfect for specifying the rules.

We still need to encode everything down to binary (ascii, unicode, bcd, floating points, pixel formats, PCM,…) and use a programming language (as defined by a grammar) to get the computer to do anything. Inference is made possible by those two mechanisms. It’s not a new computation model.

Post reply on HN