Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 vs. Opus

techstackups.com

Post reply on HN
Print view View original

311–320 of 367 posts

Re: GLM 5.2 vs. Opus

#311

Post by w0m »

wiremine wrote: post #290
cmrdporcupine wrote: post #286

Earlier quoted context omitted.

Not really, GLM uses more tokens to get work done.

I ran a fairly large experiment last week, and the token usage wasn't bad at all. What softs of use cases are you seeing large token usage by GLM 5.2?

> are you seeing large token usage by GLM 5.2

the statement isn't "GLM 5.2 has large token usage", it's "GLM 5.2 has large token usage vs modern Opus".

I haven't used it, but this wouldn't surprise me. I see ~30% lower token usage for better results with Opus 4.8 vs 4.6 (and i had great results with 4.6)

Re: GLM 5.2 vs. Opus

#312

Post by nileshtrivedi »

sanderjd wrote: post #197
post-it wrote: post #180

Earlier quoted context omitted.

The streetlight effect: > A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies, no, and that he lost them in the park. The policeman asks why he is searching here, and the drunk replies, "this is…

Sure, for casual evaluation, I agree. But are there serious analyses that are evaluating this kind of thing? I mean, these are the kinds of things I evaluate in my own work when a new model comes out, or when I'm evaluating a harness. But this is all very ad hoc and intuitional. I'd love to start bringing rigor to it, but I haven't found much prior art on this. In another thread someone said that's because it's proba…

You might like SWE-WebDevBench which tries to do this comprehensive evals for webapp development. https://webdevbench.com/

Re: GLM 5.2 vs. Opus

#313

Post by kgeist »

CuriouslyC wrote: post #131
em500 wrote: post #43

Earlier quoted context omitted.

We've had the great small Qwen 3.6 early April that many could actually run on their laptop. Then similar from Google a few weeks later (Gemma4, better in prose, worse in code). Then the super cheap large Deepseek V4 a few weeks later. Then antirez DS4 build that made that actually runnable on MacBooks and Mac Studios. And now the "near-frontier / near-Opus" GLM 5.2. For people who follow open LLMs, none of these wer…

The difference here is that those small models are impressive, but not super useful. Deepseek 4 is impressively cheap for the intelligence, but not reliable enough to daily drive unless your time has low value. GLM passes a meaningful threshold of reliability/utility that puts it in a different category for real work. Just like Opus really took off after passing a threshold with 4.5. It's the first open model to do t…

Qwen3.6-27b is surprisingly good for tasks that need modifying an existing repo by analogy with the existing code. For example, you have an existing CRUD app and want to add a new domain model and expose it via the API. Qwen3.6 analyzes how things are done in the project and usually makes it work flawlessly in one shot, and the code is what you expected more-less. Qwen3.6 only struggles with non-trivial code or when you bootstrap a project from scratch (due to the lack of world knowledge, it's a small model after all). But how often do you write non-trivial code or projects from scratch?

I once gave Sonnet 4.6 and Qwen 3.6 the same real-world task to compare: "extend the existing code with this new requirement". Qwen3.6-27b perfectly followed the existing conventions, while Sonnet 4.6 invented its own conventions that were rejected during CR by another dev (i.e. he basically chose Qwen3.6's output in a blind test). Qwen3.6-27b, run locally, also managed to finish faster on that task (mostly because Sonnet 4.6 made tool calling errors and removed some code by accident, so it spent additional time reverting its errors, and got somewhat confused in the process).

We already have production code running live that was written entirely by Qwen3.6-27b. Although, we plan to move to self-hosting GLM5.2 because it's more versatile.

Re: GLM 5.2 vs. Opus

#314

Post by nijave »

david_shi wrote: post #24

> GLM-5.2 cost a fraction as much. Opus finished in half the time and shipped a cleaner game. Off topic, but does anyone else instantly pick up on LLMisms like this? It seems like all the models have converged on this style of writing, and improvements aren't really changing it.

Yes, especially if I know the person/have heard them talk before.

Re: GLM 5.2 vs. Opus

#315

Post by spider-mario »

nfriedly wrote: post #285
ijidak wrote: post #248

Earlier quoted context omitted.

What type of task are you running for ten hours? Is this a programming task? I've not come across a programming task that would take an LLM ten hours.

I'm not the person you asked, but if they're running in their own local hardware, then it might just be a lot slower than what the big providers run their models on. System RAM is a lot cheaper than VRAM, especially if you bought it last year.

They said it’s GPT 5.5.

Re: GLM 5.2 vs. Opus

#316

Post by throwawayffffas »

cultofmetatron wrote: post #19

I seriously dont' know all this big hullabaloo about one shot prompting. by definition, a single prompt wont' constitute the complexity of a software project. ergo, what you'll get is a series of assumptions made by the model based on preexisting code in its training corpus. I'd rather see a coding agent that can follow steps in a plan file to a T while following guardrails and adhering to the proper coding conventio…

While I generally agree, there is correlation, will it go into doom loops? Will it overthink obvious bugs? Will it hallucinate libraries and apis? If it does all that in the one shot prompt it will do it in your codebase as well. Yeah state of the art models won't do all that. But that was not the case only 6 months ago, and doing the one shot prompting demonstrates ti does not do all of the above. It's a quick first real easy to clear test, really.

Re: GLM 5.2 vs. Opus

#317

Post by throwawayffffas »

somenameforme wrote: post #223
cultofmetatron wrote: post #19

I seriously dont' know all this big hullabaloo about one shot prompting. by definition, a single prompt wont' constitute the complexity of a software project. ergo, what you'll get is a series of assumptions made by the model based on preexisting code in its training corpus. I'd rather see a coding agent that can follow steps in a plan file to a T while following guardrails and adhering to the proper coding conventio…

Unless I'm missing something, the prompt he gave must have been fairly detailed because both games are basically identical. But for a more practical issue, the ultimate goal of LLMs is to replace software engineers, or at least enable everybody to become a software engineer, to use a more up-beat phrasing that's no less accurate. And so an LLM's ability to reliably construct something from a poorly defined, contradic…

It's the same prompt but they also gave them the same asset pack.

Re: GLM 5.2 vs. Opus

#318

Post by jonathanlydall »

cultofmetatron wrote: post #190
scwoodal wrote: post #104

Earlier quoted context omitted.

> I'd rather see a coding agent that can follow steps in a plan file to a T while following guardrails and adhering to the proper coding conventions in the human reviewed spec. Guardrails/conventions should be enforced in linters, formatters, static analysis tooling; not specs/prompts.

lets say you have a table that is partitioned. how do you lint/format "any select into this table MUST include the partition key in the predicate and any join must include it in the on." I'm not personally familiar with any static analysis tool that does this but its trivial to implement with an llm prompt. trivially easy to add to your automated PR reviews.

In a C# project I worked on in the past we used a specification pattern in front of EF to achieve essentially statically verifiable rules like this.

The specification pattern is essentially a builder for a query (or more accurately in our case, for a C# expression for the ORM) and one of the main benefits is that you can put rules into the “build” method which returns the expression to be used, such as “throw an exception if the WithPartionKey() method hasn’t been called”.

Usual ORM disclaimer applies that you should double check the generated SQL is acceptably performant. The specification pattern can make this better or worse, it can result in contorted expressions which result in poorly performing SQL, but you could also put logic in your specification to “ensure” good SQL, e.g. by having a method which uses the correct expression structure which is known to generate good SQL (e.g. EF6 would generate terrible SQL for certain nested Any()’s, but if you used .Contains() instead it was fine) and the encapsulation means that all callers of the method get the benefit.

It worked well for us as reviewing the specification methods in PRs was pretty easy and less experienced developers on the project were more likely to do the performant and working thing than not.

Re: GLM 5.2 vs. Opus

#319

Post by throwawayffffas »

cmrdporcupine wrote: post #286
faxmeyourcode wrote: post #274

I feel like another comparison worth looking at is purely cost. Capability per dollar is something I care about: Opus API $5/$25 Sonnet API $5/$15 Haiku API $1/$5 GLM 5.2 API $1.4/$4.4 So you're really getting near opus level capability for the price of haiku .

Not really, GLM uses more tokens to get work done.

In the article, they claim GLM used almost half the tokens 131,000, and the cost is about a quarter. For the cost to be the same GLM would have to use 4-5 times more tokens.

Re: GLM 5.2 vs. Opus

#320

Post by noonething »

digitaltrees wrote: post #299
wahnfrieden wrote: post #280

Earlier quoted context omitted.

They are absolutely not good enough

I disagree because I am getting real work done. But I have three Mac studios with 1.5 tb of ram and built my own harness around large models in my own ide (propelcode.app). It’s not perfect. And opus ChatGPT 5.5 are better but they’re good enough

if you had the information to do the task perfectly then those open models should be fine no? if so, your just paying for knowledge at that point.
Post reply on HN