Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 vs. Opus

techstackups.com

Post reply on HN
Print view View original

51–60 of 367 posts

Re: GLM 5.2 vs. Opus

#51

Post by halyconWays »

cultofmetatron wrote: post #19

I seriously dont' know all this big hullabaloo about one shot prompting. by definition, a single prompt wont' constitute the complexity of a software project. ergo, what you'll get is a series of assumptions made by the model based on preexisting code in its training corpus. I'd rather see a coding agent that can follow steps in a plan file to a T while following guardrails and adhering to the proper coding conventio…

"We did multi-shot prompting to try and get these two games into comparable states using these two different models."

"Well obviously you provided better follow-up prompts to the one that came out better."

Also nothing about human-provided plan files and guardrails preclude the one-shot benchmark test. Heavens, I almost said "real coding," but in "real agentic program creation" you'd obviously be doing multi-turn interaction with the agent, but how can you provide a fair test when the model's output n determines your n+1 response?

Re: GLM 5.2 vs. Opus

#52

Post by _pdp_ »

In the name of science we crafted an autonomous AI agent that builds games on a loop. It is based on GLM 5.2.

I am not sure where this is going to lead us but it is fun to watch.

Re: GLM 5.2 vs. Opus

#53

Post by Oras »

xlii wrote: post #6

I've been checking out GLM 5.2 on some projects and few thoughts on it: - it takes it sweet time to get code rolling, not the fastest model by any means - it strays a lot during discovery/planning but then corrects - it's not steering friendly, as it hallucinates things that it doesn't follow later on - its output is quite good A sample use case: I was optimizing rendering on Swift+Zig codebase. It chocked on 5k data…

Also pricing, I wanted to give a try, but when pricing is only 30% cheaper than Opus, I wouldn't go for it with these issues.

Re: GLM 5.2 vs. Opus

#54

Post by cicko »

jack_pp wrote: post #33
ulrikrasmussen wrote: post #30

> Through an API it costs a fraction of Opus, and you can run it yourself for free if you have the hardware. I haven't been keeping up on hardware costs for state of the art LLM inference, but this remark made me ask myself how many readers of the article would actually be able to run this model on hardware they own. How much would it cost to acquire such a setup?

This framing local LLMs as free is stupid. Basically pay 100+ months worth of API costs up front isn't free in the slightest. And it will be slower than non-local, your hardware will be outdated in 12 months and probably won't be able to run SOTA at anywhere near non-local speed in max 20 months

True. But there are other meanings of "free". I.e. nobody can say "from now on you no longer have access to model X because you're an asshole"

Re: GLM 5.2 vs. Opus

#56

Post by [deleted] »

halyconWays wrote: post #51
cultofmetatron wrote: post #19

I seriously dont' know all this big hullabaloo about one shot prompting. by definition, a single prompt wont' constitute the complexity of a software project. ergo, what you'll get is a series of assumptions made by the model based on preexisting code in its training corpus. I'd rather see a coding agent that can follow steps in a plan file to a T while following guardrails and adhering to the proper coding conventio…

"We did multi-shot prompting to try and get these two games into comparable states using these two different models." "Well obviously you provided better follow-up prompts to the one that came out better." Also nothing about human-provided plan files and guardrails preclude the one-shot benchmark test. Heavens, I almost said "real coding," but in "real agentic program creation" you'd obviously be doing multi-turn int…

[deleted]

Re: GLM 5.2 vs. Opus

#57

Post by wejick »

Totally agree witg the general assessment. The biggest problem with Z.ai model for a long time is not quality, but the inference speed and general capacity availability. Hopefully with this recent hype, there will be more provider on openrouter for 5.2.

Re: GLM 5.2 vs. Opus

#59

Post by HPsquared »

jofzar wrote: post #50
TurdF3rguson wrote: post #35

Pretty clearly it's beating Opus at [web dev]( https://www.gptbased.com/ ) - on price, on score.. I mean what else is there?

I hate to be that guy, but real privacy policy on training data/it being hosted somewhere where I'm not worried about secrets being stored/leaked.

Open weights win on that front surely?

Re: GLM 5.2 vs. Opus

#60

Post by jaapz »

cultofmetatron wrote: post #19

I seriously dont' know all this big hullabaloo about one shot prompting. by definition, a single prompt wont' constitute the complexity of a software project. ergo, what you'll get is a series of assumptions made by the model based on preexisting code in its training corpus. I'd rather see a coding agent that can follow steps in a plan file to a T while following guardrails and adhering to the proper coding conventio…

When the model produces reasonable results from one prompt, you could assume that it will also return reasonable results through the follow up prompts.
Post reply on HN