Live data from Hacker News

Local Qwen isn't a worse Opus, it's a different tool

blog.alexellis.io

181–190 of 271 posts

Re: Local Qwen isn't a worse Opus, it's a different tool

#181

I feel like it's the Emperor's new clothes reading this article and seeing the praise it's getting. This sentence doesn't even make sense: > These products use very low level Linux primitives like containers, Kubernetes, Firecracker microVMs, and networked protocols. Out of anything that is a "low level linux primitive" I could maybe argue that networking? protocols fit the bill. And it's obviously fully AI-generated…

Low level today means JavaScript instead of typescript

Re: Local Qwen isn't a worse Opus, it's a different tool

#182

Earlier quoted context omitted.

> If you have a toolbox full of similar but different tool getting to know them is a prudent thing to do, not a psychosis Can be both. Use of some tools like LLMs might be more inducing psychosis than others like plain compilers or hammers. > And if humans are anything, they are tool users. To the point of self-destruction sometimes.

> Use of some tools like LLMs might be more inducing psychosis than others like plain compilers or hammers. I really don't get it. Why the fact that it outputs words is so goddamn important for everybody? How does it suddenly make you so emotionally vulnerable? Does my brain work in a different way than the rest of humanity? Can't you disregard what's irrelevant? Is every programmer suddenly a trump supporter that ha…

It's a certain percentage of people and yes it's different for them because it outputs words and triggers some kind of emotional trust response.

Re: Local Qwen isn't a worse Opus, it's a different tool

#183

Earlier quoted context omitted.

> I would love to have a product sheet showing what each models strengths an weaknesses are, so that I can have a clear decision tree of "if this kind of work, use model X", or "model Y should be used in ways Z". But they all look the same from the outside and the only way to figure out which might be marginally better at what is to do extensive, time consuming, and perhaps expensive testing. Think of it less like a…

They are not human. Humans have names, faces, voices, personality, a personal history, family, care for whatever they call their community. With humans it's actually good and worthwhile to create and strengthen connections. With an LLM, that's psychosis.

They are not human, but it helps to prompt them similarly. See: https://www.anthropic.com/research/emotion-concepts-function

Re: Local Qwen isn't a worse Opus, it's a different tool

#184

Earlier quoted context omitted.

A poor analogy depending on the setting because you can't adjust the odds with a slot machine, and the ROI is negative by design. If that's your experience, yeah, I wouldn't use an LLM either.

Pretty sure most modern slot machines are digital and you could adjust the odds (even to a positive EV) if you change the code.

You're being unfaithful to the original statement. The whole point of saying something is like a slot machine is that there are significant odds that you lose. If you ever have access to a casino slot machine that has a positive EV, there are no tangible negative aspects anymore; you would use it over and over again and accumulate significant wealth from the house. That's my point.

Re: Local Qwen isn't a worse Opus, it's a different tool

#185
Tangential question (since they brought it up in the article) from someone not involved in AI performance optimization:

How big of a deal is looping, practically? Or, I mean, I see thinking models loop occasionally. But it seems to me that every token in the loop should be in the KV cache already, is there really no way to either power through a loop because of the 100% cache hit rate, or identify that you are in a loop that way? (As a human, when thinking hard I sometimes loop, but it is easy enough to identify…)

Re: Local Qwen isn't a worse Opus, it's a different tool

#186
post #153
post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

I asked GLM 5.2 for a HTML5 port of my old C#/XNA game. It ported all the code exactly (except for operator overloading, which doesn't exist in JS), and added more code to make the code work. I asked Claude Sonnet 4.6 for the same thing, and Claude's version was like if the game had been written in JS originally. Also, for some reason it made it a single HTML file, removed all assets, dynamically generated graphics a…

You’d probably have to say “port exactly as is without changing any assets and keeping the original structure of the code” or “port with using the exact same assets but write as if native JS but use good code structure principles for organizing”.

You have to be a lot more explicit but it’s hard to know a priori what decisions it’ll make. A good idea is to run it in plan mode so you can read those decisions before it sets out on a path and have an opportunity to make corrections.

Re: Local Qwen isn't a worse Opus, it's a different tool

#187

Earlier quoted context omitted.

I think this goes beyond "vibes" to cargo-culting. It's why nobody's ever able to actually show ROI from LLMs

It's hard to actually show ROI from any programming methodology or tool. You can show ROI from a product or feature, but the tool/methodology is a multiplier on the velocity of creating that which is not directly observable.

It's really not. When we switched from CVS to SVN I had to show ROI and when I we switched from SVN to git I had to show ROI and when we switched from Ada to Java I had to show ROI. When we switched from Xen to KVM I had to show ROI and when we switched from PAM realtime privileges to rtkit I had to show ROI. When we switched from chroots to LXC I had to show ROI, when we switched from LXC to docker I had to show ROI, and when we switched from docker to podman I had to show ROI.

If you can't show ROI there's literally no reason to ever switch anything.

Re: Local Qwen isn't a worse Opus, it's a different tool

#188

Earlier quoted context omitted.

Right, this is why I would slam the breaks on investing into your workflow all of your time and effort, because 2 months from now it may be out the window. Frontier models are also constantly being tweaked, so what worked yesterday may be off today. ChatGPT was obedient with the grill-me technique, just wrote a plan. Yesterday it started jumping to implementation. Why?

I find that when an LLM jumps into tasks it was not told to do (or even worse, doing things it was explicitly told not to), it is a good sign the context is too full, and you should do a controlled hand-off to a new instance.

I wipe my context relentlessly. I never have long-running conversations. In and out like Seal Team Six.

Re: Local Qwen isn't a worse Opus, it's a different tool

#189

Earlier quoted context omitted.

> We’ve been calling some of these “magic words” at work, specific technical terms or references/techniques that you need only mention to get vast improvements in outcome. Any chance you could share some of these? Seems like something we could all benefit from.

Sure, my company has been working on a broad swathe of infrastructure projects and developer tools, which requires prompting models to seek out other tools/apis/docs/examples but in a way where we can't just dump all the context on the model up front. We also need the models to oftentimes look up technical documentation and specs, and sometimes build custom parsers for specific documentation websites that only make t…

Thank you very much. I'm going to re-read this evening. Have a great day!

Re: Local Qwen isn't a worse Opus, it's a different tool

#190
post #6

If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument. With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative.…

> you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative

This has been my experience with most models. If you say "How do I do X? I was thinking maybe Y or Z" then the model will probably try to make Y or Z work. They will very likely not say some third option that is wildly different is better, even if it may be. And actually maybe less so with Claude because sometimes it pushes back.

Actually this seems like it would be an interesting test. Maybe I will come up with some contrived question and ask several models.

Post reply on HN