Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.2 is a step change for open agents

interconnects.ai

Post reply on HN
Print view View original

91–100 of 240 posts

Re: GLM-5.2 is a step change for open agents

#91

Post by JSR_FDED »

verdverm wrote: post #75
mlmonkey wrote: post #61

Here are the numbers from their bar chart: 1. SWE-bench Pro Model Score (%) GLM-5.2 62.1 GLM-5.1 58.4 Claude Opus 4.8 69.2 GPT-5.5 58.6 Gemini 3.1 Pro 54.2 2. Terminal-Bench 2.1 Model Score (%) GLM-5.2 81.0 GLM-5.1 63.5 Claude Opus 4.8 85.0 GPT-5.5 84.0 Gemini 3.1 Pro 74.0 3. NL2Repo Model Score (%) GLM-5.2 48.9 GLM-5.1 42.7 Claude Opus 4.8 69.7 GPT-5.5 50.7 Gemini 3.1 Pro 33.4 4. DeepSWE Model Score (%) GLM-5.2 46.2…

copying the graphs and tables to HN is noisy and harder to read

Still more helpful than this comment

Re: GLM-5.2 is a step change for open agents

#92

Post by ukuina »

timcobb wrote: post #21

Can people share their GLM and open model setups in general please? What provider do you use. Why do you trust it with serving full quality? What harness do you use? Why do you trust it not to have malware (most harnessed are TS apps). I am just trying GLM 5.1 from Nvidia build in open code would love to hear how you all do it, thanks.

Synthetic.new and Claude Code using GLM-5.2. Great model, but the harness will error out if using subagents. The base plan only allows one concurrent request at a time. Also, GLM will burn through your weekly quota in a day if you're not precise with your scope.

Re: GLM-5.2 is a step change for open agents

#94

Post by Fr0styMatt88 »

timcobb wrote: post #21

Can people share their GLM and open model setups in general please? What provider do you use. Why do you trust it with serving full quality? What harness do you use? Why do you trust it not to have malware (most harnessed are TS apps). I am just trying GLM 5.1 from Nvidia build in open code would love to hear how you all do it, thanks.

Local using Qwen3.6-27B; 2xRTX 5070Ti graphics cards; VS Code with Cline at the moment and Ollama back-end (will get to trying the others soon).

Re: GLM-5.2 is a step change for open agents

#95

Post by teravor »

dools wrote: post #57
themgt wrote: post #11

I just tested GLM 5.2 out via Z.ai in pi for a little one-off project that was already scoped. It actually did a relatively decent job starting out, and figured important things out from context. But the reasoning traces became increasingly hilarious, with it getting confused and going in loops, doubting itself. I began to feel almost sad, it was like listening to the internal monologue of someone with anxiety disord…

The reasoning traces always look terrible and they’re frustrating to watch. It’s the same with Kimi. What’s interesting is that the end result is then good. I think it’s just some sort of devils advocate trick to get better output.

as compared to what though? you can't see the actual think traces for opus or gpt.

Re: GLM-5.2 is a step change for open agents

#97

Post by [deleted] »

mlmonkey wrote: post #61

Here are the numbers from their bar chart: 1. SWE-bench Pro Model Score (%) GLM-5.2 62.1 GLM-5.1 58.4 Claude Opus 4.8 69.2 GPT-5.5 58.6 Gemini 3.1 Pro 54.2 2. Terminal-Bench 2.1 Model Score (%) GLM-5.2 81.0 GLM-5.1 63.5 Claude Opus 4.8 85.0 GPT-5.5 84.0 Gemini 3.1 Pro 74.0 3. NL2Repo Model Score (%) GLM-5.2 48.9 GLM-5.1 42.7 Claude Opus 4.8 69.7 GPT-5.5 50.7 Gemini 3.1 Pro 33.4 4. DeepSWE Model Score (%) GLM-5.2 46.2…

[deleted]

Re: GLM-5.2 is a step change for open agents

#98

Post by JSR_FDED »

fraywing wrote: post #46

It feels like the gap is closing from an intelligence perspective. Or at least doing some kind of log flattening. Been playing with GLM 5.2 in different contexts. It's less good if you don't max out thinking, but as xhigh it's been able to solve most problems I was throwing at Opus in the about the same amount of time (via OpenRouter). Wild time to be alive.

Anecdote, not “research”:

Yesterday I compared Deepseek, Kimi 2.6, MiMo 2.5 and GLM 5.2 for the same task (replace a custom token-based auth scheme with a cookies-based scheme across a front- and back-end codebase).

I used Opencode with the zen subscription to try different models.

All did this perfectly, basically indistinguishable from each other. However, when I pointed out that the new cookies-based auth didn’t allow multiple independent logins across browser tabs (which the previous scheme did allow) I noticed this:

Deepseek, Kimi, MiMo started giving me multiple options but advocating strongly that I should either accept this deficiency, or don’t use the cookies version (keep the old auth scheme). They were so similar it was as if they were all the same model.

Only GLM 5.2 said “here’s how to use cookies and also have tab-level separation”. The difference vs the other models was very stark.

Re: GLM-5.2 is a step change for open agents

#99

Post by jimbokun »

IncreasePosts wrote: post #68
matheusmoreira wrote: post #55

Earlier quoted context omitted.

Yeah. There's no way to verify what these providers are doing. The real future is running these models at home. Opus level inference on our own hardware would be a dream come true.

How will anyone running home instances be able to compete against people paying some money running much more powerful models on much more powerful hardware?

At some point it will be hard for us to tell the difference.

Re: GLM-5.2 is a step change for open agents

#100

Post by fc417fc802 »

rufo wrote: post #90
dools wrote: post #57

Earlier quoted context omitted.

The reasoning traces always look terrible and they’re frustrating to watch. It’s the same with Kimi. What’s interesting is that the end result is then good. I think it’s just some sort of devils advocate trick to get better output.

The reasoning tokens are really just there to extend the amount the LLM can "compute" the problem; put another way, the only way a given model can "think" more about a problem is to fill more of its context with predicted tokens, which has the effect of increasing the accuracy of each token. The reinforcement learning these models go through generally doesn't care what the chain of thought tokens look like (outside o…

For me what really drove this point home (that reasoning traces aren't "real" by any reasonable definition of the term) was noticing instances of things being out of order and exhibiting various inconsistencies with the final answer. My favorite was an example posted to HN that went something along the lines of the model first output the conclusion, then performed the supposed derivation after the fact, then stated it needed to verify the earlier conclusion to verify the derivation was correct so it hallucinated a tool call, then it remarked positively about the verification matching, and finally it output a slightly different answer. At no point was the answer actually correct although it was vaguely in the ballpark.
Post reply on HN