Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.2 is the new leading open weights model on Artificial Analysis

artificialanalysis.ai

Post reply on HN
Print view View original

331–340 of 476 posts

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#331

Post by meatmanek »

robertkarl wrote: post #231
vorticalbox wrote: post #29

Earlier quoted context omitted.

This is a problem I find with opus is will spend so long thinking then going “but wait what if” To point where I stop it and simple tell it to “start writing code you can work it out as you go along” Seems writers block also effects LLM

https://arxiv.org/abs/2606.00206 In this paper they nerf an LLMs ability to emit waffling thinking tokens like "wait", "but", "alternatively", and the models (they're old, small models in the paper) terminate reasoning faster and perform better. I bet Anthropic is tuning this on their backend.

This is super cool. Do you know if any of the inference backends (llama.cpp, vllm, etc) support this technique?

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#332

Post by granra »

eckelhesten wrote: post #328
granra wrote: post #312

Earlier quoted context omitted.

That's what I've been doing. I use crush normally. While the codebase are by no means huge, they're not tiny either.

Are you using it in an agentic workflow? Just reading the codebase will consume a lot of cached tokens, but seemingly, z.ai counts these as normal input tokens the way they're rate limiting.

I'm not entirely sure what an agentic workflow could mean today but I think so. I use a coding agent (crush), prompt it to brainstorm an implementation with me (or sometimes I know exactly how I want to implement it but ask it to challenge it), correct any wrong assumptions or request the implementation to look differently than suggested if I don't like it. Then finally when I'm positive I've cleared the most important assumptions I ask it to actually write and edit files and run tests and such (this just ends up being a "implement this").

With any model I've tried I've found it to be a huge pain to have it fix things where it made a wrong assumption without the code becoming a mess and burning a lot of tokens. I'm aware that not everyone works like this but I'm still very opinionated on what the end result should look like so I can still work on it without an LLM.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#333

Post by alexjplant »

Tiberium wrote: post #2

It seems to really be a nice step-up and is getting quite close to the frontier. I wish they'd start focusing on the reasoning efficiency now, though. I have a simple (relatively) test task to evaluate LLMs: writing a simple math evaluator library in Nim (it's about 400-600 lines total max), and GLM 5.2 (xhigh which maps to max effort) spent over 15 minutes (!) reasoning, spending about 45k tokens, before it finally…

> It seems to really be a nice step-up and is getting quite close to the frontier.

IMHO it's already surpassed them. I vastly prefer my personal GLM and OpenCode setup to the Claude Code and Opus one that I have to use at work. The former makes way fewer StackOverflow brogrammer-tier mistakes and is considerably better at following instructions. The harness UX is also vastly superior as it doesn't ignore, randomly change, or incorrectly report settings.

Maybe it's the harness and I'd have even greater success with OpenCode and Anthropic, but I think it safe to say that Anthropic's moat is evaporating.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#334

Post by egeozcan »

cromka wrote: post #258
Schiendelman wrote: post #37

Earlier quoted context omitted.

Sure, I'm not saying I, a software engineer, cannot do this. I'm saying it's significant onboarding friction . Unless this were a massive differentiator, people aren't going to be "talking about it" the way GP suggests!

> it's significant onboarding friction. It's crazy that apparently writing software without knowing how to edit a single config file is normal now.

For me it's about tolerance. When I was 13, I could and would customize everything, so much that the computer repair shop told my father that their son "likely is a hacker or something".

At 40, I could easily configure claude code to use another model, even if there weren't any official guides with a bit of MITM fun, but I don't want to invest my attention / heavily use something that will most likely break in the near future.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#335

Post by bitmasher9 »

zozbot234 wrote: post #311
program_whiz wrote: post #263

Earlier quoted context omitted.

"approaching" is doing some work there. $30K today will get you 90-144GB usable VRAM with solid system RAM and disk and CPU. A single B200 chip at 180GB is $40K. Unfortunately that is nowhere close to being able to run a 750B param model. For something like that, we're getting closer to 1TB VRAM (8+ H200/B200), and then 1M context KV cache is many more GBs on top of that. That's a $500K-$1M+ rig as of now. That's a l…

> Unfortunately that is nowhere close to being able to run a 750B param model. For something like that, we're getting closer to 1TB VRAM You don't have to run a model from VRAM, or even from a sizeable amount of RAM. These choices only ever make sense when serving the model at scale, to hundreds of simultaneous users or more.

For workstation inference a unified memory architecture would be a good cost/performance balance, while keeping COGs reasonable.

512GB unified memory macs are available, with the ram upgrade costing a few grand.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#336

Post by [deleted] »

applicative wrote: post #327
JKCalhoun wrote: post #232

Earlier quoted context omitted.

I would imagine it will be a fundamental breakthrough, not weights alone, that are going to usher in the next generation of AI. Perhaps China will in fact make that breakthrough. They certainly seem to have a lot of eyeballs in the field right now.

There has really been one break-through, the actual construction of giant LLMs from the available titanic corpus of text. Even that barely involved much conceptual breakthrough, a few things maybe e.g. transformer. Basically it was a question of the accessibility of a) giant internet corpus of actual people actually saying stuff and b) adequate computing power. The witty surface training, the scaffolding for a chatbo…

[deleted]

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#337

Post by bigyabai »

guybedo wrote: post #323

It's probably a good model but they used GLM 5.1 to code their infra. I signed up to their max plan yesterday, did some light coding work, and i'm at 180M tokens used and 40% weekly quota gone. Even when tokenmaxxing on the Claude Max or GPT $200 plan, i couldn't get more than 20% quota gone per day.

Are you using it for long context windows? I burn through my 5hr quota with GLM almost instantly on 200k+ contexts, but if I reset every ~100k or so it's much more manageable.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#338

Post by zackify »

daniban wrote: post #329

I'm curious what harness everyone is using for these? I want to start to test some of these open models but don't know what tools people use to get these working "agenticaly"

pi.dev and ask ai to add features you miss from claude or codex. i configure keyboard shortcuts and swap models easily

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#339

Post by conception »

vadansky wrote: post #230
lukewarm707 wrote: post #179

Earlier quoted context omitted.

with open models you can get a subscription with privacy, at the same cost as codex. openai, google and anthropic subscriptions are not available with privacy. looking at the link there it's interesting that going from cursor cli to codex cli take gpt 5.5 from 7th to 3rd. but they didn't do open model in codex. so, hard to say it's for sure a model benchmark. maybe open models are just shit at swe agent harness...it'…

> with open models you can get a subscription with privacy Unless you're running it locally, aren't you just trusting some other entity?

While true - there are laws about saying you are doing the things you are doing, especially in certain regulated environments. If you are in the same country as the entity you are trusting, you have recourse if they are not living up to your trust usually in some form or another.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#340

Post by celrod »

epolanski wrote: post #100
RyanHamilton wrote: post #96

Earlier quoted context omitted.

Could it be possible, these firms are optimizing for two things: a) Better performance. b) Gathering data from you to further improve performance later. I've also found the huge amount of planning rather than iteration frustrating. I've felt like I'm teaching a junior!

I think they simply optimize around E2E benchmarks, none of those benchmarks is designed as multi turn assistance to the user, but going from a prompt straight to the final solution.

Exactly. How can "we" develop and encourage benchmarks for multi-turn user assistance? That is what I want. I feel like the models and harnesses push much too hard against this workflow -- that they push you towards letting go and vibe coding, with only your discipline (and desire for a quality and maintainable product) holding it back.
Post reply on HN