Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 vs. Opus

techstackups.com

Post reply on HN
Print view View original

21–30 of 367 posts

Re: GLM 5.2 vs. Opus

#21

Post by wongarsu »

jameswhitford wrote: post #4
meander_water wrote: post #2

> So we ran it head-to-head against Claude Opus 4.8: same one-shot prompt, build a 3D platformer in raw WebGL from scratch Running a single one-shot prompt is not a benchmark, not is it representative of any sort of real-world usage. Most agent usage is collaborative so you need to test things like reliability (when I delegate a task, does it complete it without making up test results for e.g.) and steerability (does…

Hi, I am the author, I completely agree! I set out to run a vibe test on this one, not a benchmark, the real benchmarks are listed. My test shows what the models can do when both tasked with a long-running, technically difficult, one-shot task. I think your test you describe (collaborative, task delegation, task completion, TTD, steerability) is a great format for a future test that I will definitely try out.

Tbf, most of the "real benchmarks" have issues that are just as bad. Assessing LLM performance is just hard

Re: GLM 5.2 vs. Opus

#22

Post by linzhangrun »

lithiumii wrote: post #8
greyman wrote: post #5

>On output tokens, GLM-5.2 is less than a fifth the price of Opus. Opus is most expensive model in pay as you go model, but IMO fair comparison should include subscription price as well. For example when one has $100 Claude Max and use it up through the month, it might not be more expensive than GLM, or at least not 5x.

GLM has subscription plans too.

Out of stock, unavailable

Re: GLM 5.2 vs. Opus

#23

Post by Aozora7 »

I used GLM 5.0/5.1/5.2 for some projects, and for me, the area in which they lag behind frontier models the most are user interfaces. They get really close to Opus when it comes to pure algorithms, but when I need something like web application or a mobile app that looks and works well, they are very noticeably worse than even Sonnet.

Re: GLM 5.2 vs. Opus

#24

Post by david_shi »

> GLM-5.2 cost a fraction as much. Opus finished in half the time and shipped a cleaner game.

Off topic, but does anyone else instantly pick up on LLMisms like this? It seems like all the models have converged on this style of writing, and improvements aren't really changing it.

Re: GLM 5.2 vs. Opus

#25

Post by Imanari »

xlii wrote: post #6

I've been checking out GLM 5.2 on some projects and few thoughts on it: - it takes it sweet time to get code rolling, not the fastest model by any means - it strays a lot during discovery/planning but then corrects - it's not steering friendly, as it hallucinates things that it doesn't follow later on - its output is quite good A sample use case: I was optimizing rendering on Swift+Zig codebase. It chocked on 5k data…

This mirrors my experience. I have been using it in Pi. It is smart and output is good but it is not efficient in getting there.

Re: GLM 5.2 vs. Opus

#26

Post by leumon »

I've seen glm 5.2 struggle writing simple compilable c code. It might be good at web, but it's world knowledge is limited due to the small model size, making it's use quite limited in my opinion.

Re: GLM 5.2 vs. Opus

#27

Post by epolanski »

cultofmetatron wrote: post #19

I seriously dont' know all this big hullabaloo about one shot prompting. by definition, a single prompt wont' constitute the complexity of a software project. ergo, what you'll get is a series of assumptions made by the model based on preexisting code in its training corpus. I'd rather see a coding agent that can follow steps in a plan file to a T while following guardrails and adhering to the proper coding conventio…

Yet this is how virtually everybody is benchmarking and fine tuning.

Since Opus 4.6 I've seen later Anthropic models being more and more capable on one hand, but also less useful on multi turn open tasks.

It feels like with each model they are more and more prone to go "their own way" and jump into the implementation as soon as they can.

I can't but blame it on benchmarks and fine tuning around prompt-to-solution work.

Re: GLM 5.2 vs. Opus

#28

Post by speedgoose »

david_shi wrote: post #24

> GLM-5.2 cost a fraction as much. Opus finished in half the time and shipped a cleaner game. Off topic, but does anyone else instantly pick up on LLMisms like this? It seems like all the models have converged on this style of writing, and improvements aren't really changing it.

I think a bunch of real humans started to adopt the LLMs writing style.

Re: GLM 5.2 vs. Opus

#30

Post by ulrikrasmussen »

> Through an API it costs a fraction of Opus, and you can run it yourself for free if you have the hardware.

I haven't been keeping up on hardware costs for state of the art LLM inference, but this remark made me ask myself how many readers of the article would actually be able to run this model on hardware they own. How much would it cost to acquire such a setup?

Post reply on HN