Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.2 is the new leading open weights model on Artificial Analysis

artificialanalysis.ai

Post reply on HN
Print view View original

281–290 of 476 posts

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#281

Post by maxdo »

benjiro29 wrote: post #59
Tiberium wrote: post #2

It seems to really be a nice step-up and is getting quite close to the frontier. I wish they'd start focusing on the reasoning efficiency now, though. I have a simple (relatively) test task to evaluate LLMs: writing a simple math evaluator library in Nim (it's about 400-600 lines total max), and GLM 5.2 (xhigh which maps to max effort) spent over 15 minutes (!) reasoning, spending about 45k tokens, before it finally…

GLM 5.2 Max = Opus 4.8 Max in thinking behavior. The thinking chain is so similar, and so is the amount of token usage on the output. If you want reasonable token usage, you need to run it GLM 5.2 at High. There is little drop in quality from Max to High (for most tasks). And it cuts token usage by 2 a 2.5x. GLM 5.2, Max is really something you only need for complex tasks. In essence, GLM 5.2 is Opus 4.8 its little b…

looking at the score this is rather a gemini 3.5 flash competitor, yes, for cheaper, but distance to opus and fable is as big as their price diff.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#282

Post by bityard »

cromka wrote: post #258
Schiendelman wrote: post #37

Earlier quoted context omitted.

Sure, I'm not saying I, a software engineer, cannot do this. I'm saying it's significant onboarding friction . Unless this were a massive differentiator, people aren't going to be "talking about it" the way GP suggests!

> it's significant onboarding friction. It's crazy that apparently writing software without knowing how to edit a single config file is normal now.

The real question is: should the file be edited in emacs or vim?

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#283

Post by cmrdporcupine »

papersail wrote: post #273
papersail wrote: post #130

Earlier quoted context omitted.

score age size name 62.0 8 - Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) 59.1 55 - GPT-5.5 (xhigh) 58.5 55 - GPT-5.5 (high) 57.2 104 - GPT-5.4 (xhigh) 56.7 20 - Claude Opus 4.8 (Adaptive Reasoning, Max Effort) 56.2 55 - GPT-5.5 (medium) 55.5 118 - Gemini 3.1 Pro Preview 53.1 132 - GPT-5.3 Codex (xhigh) 53.1 62 - Claude Opus 4.7 (Non-reasoning, High Effort) 52.5 62 - Claude Opus 4.7 (Adaptive Re…

rank score age size name 1 62.0 8 - Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) 2 59.1 55 - GPT-5.5 (xhigh) 3 58.5 55 - GPT-5.5 (high) 4 57.2 104 - GPT-5.4 (xhigh) 5 56.7 20 - Claude Opus 4.8 (Adaptive Reasoning, Max Effort) 6 55.5 118 - Gemini 3.1 Pro Preview 7 53.1 62 - Claude Opus 4.7 (Non-reasoning, High Effort) 8 53.1 132 - GPT-5.3 Codex (xhigh) 9 52.5 62 - Claude Opus 4.7 (Adaptive Reason…

My observations:

Surprised to see MiniMax M3 so low on that list, not really my experience, I found it smarter than Gemini for a lot of things, that's for sure.

Also surprised to see Gemini 3.1 ranked that high there. It remains IMHO blatantly incompetent for tool use even in their own harnesses, so I can only assume this benchmark isn't ranking workflow things very high. Gemini can write code just fine. It just can't work well as an agent.

GLM 5.2 and Qwen3.7 max were from my experience fairly expensive to use on a per token price and hard to argue in favour of when the SOTA coding plans have a fixed price that makes them potentially more cost effective. (Yes I know z.ai has a coding plan but I've heard reliability nightmare stories, and it's not very cheap)

DeepSeek is clearly the best value for $$. With the right harness and prompting.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#284

Post by christoff12 »

papersail wrote: post #130
kristopolous wrote: post #104

I have a script that ranks these based on codingindex from Artificial Analysis. All it does is pull a json from their main table page and parses it with the fields I care about (coding). There used to be a mailing list associated with it but eh ... there wasn't much interest. I use the script every day though. Current partial output score age size name 47.1 58 large Kimi K2.6 47.5 54 large DeepSeek V4 Pro (Reasoning,…

score age size name 62.0 8 - Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) 59.1 55 - GPT-5.5 (xhigh) 58.5 55 - GPT-5.5 (high) 57.2 104 - GPT-5.4 (xhigh) 56.7 20 - Claude Opus 4.8 (Adaptive Reasoning, Max Effort) 56.2 55 - GPT-5.5 (medium) 55.5 118 - Gemini 3.1 Pro Preview 53.1 132 - GPT-5.3 Codex (xhigh) 53.1 62 - Claude Opus 4.7 (Non-reasoning, High Effort) 52.5 62 - Claude Opus 4.7 (Adaptive Re…

Lol thank you for sorting.

Are the scores here normalized such that each point difference is equidistant?

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#285

Post by [deleted] »

Tiberium wrote: post #2

It seems to really be a nice step-up and is getting quite close to the frontier. I wish they'd start focusing on the reasoning efficiency now, though. I have a simple (relatively) test task to evaluate LLMs: writing a simple math evaluator library in Nim (it's about 400-600 lines total max), and GLM 5.2 (xhigh which maps to max effort) spent over 15 minutes (!) reasoning, spending about 45k tokens, before it finally…

[deleted]

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#286

Post by JSR_FDED »

tekchip wrote: post #194
kristopolous wrote: post #162

Earlier quoted context omitted.

I'm actually more curious about IBM. Their granite series appears to be nowhere close to competitive. They had Watson, remember, it won on jeopardy like 15 years ago? They've been at this for a long time Maybe it's good at something else?

IBM doesn't do technology they do contracts. Any "technology" is marketing stunts. They hire a bunch of "fellows" outside contractors to make a thing they can be first at or whatever, do the stunt, then get a bunch of 5-10 year contracts with customers off the stunt. They then fuck it up for that length of time but still get paid due to those contracts. After that space of time the folks theyve burned have moved on,…

Don’t forget the marketing for the new $1B “initiative” (fill in: mobile, cloud, blockchain, AI,…)

Upon closer inspection the $1B is (a) over 10 years, (b) mostly internal cross-billing between departments.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#287

Post by hereme888 »

adastra22 wrote: post #271
hereme888 wrote: post #266

Hmmm... GLM insists it's Gemini. https://github.com/zai-org/GLM-5/issues/79

Then why does it score better than any Gemini model?

As I understand, some people tend to "distill" LLM models. Google hasn't released a new Pro version in a while. I'm not an expert in LLMs.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#288

Post by JSR_FDED »

JKCalhoun wrote: post #229
tcp_handshaker wrote: post #137

Earlier quoted context omitted.

Short comments... - GPT 5.5 consistently the best, an opinion who gets me constant downvotes here by the Anthropic Marketeer strike force... - China is going to eat the US lunch on AI - What have European universities and companies been doing? Its like if, on a parallel past/future, Nikola Tesla and Edison would have created flying Cyberpunk machines, while Europeans researchers, would be getting together to request…

"…Anthropic Marketeer strike force…" Might also just be the result of "good will" (that the company has deftly fostered). Other companies might learn from Anthropic in that regard.

“Good will” is easier if OpenAI is your yardstick

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#289

Post by bityard »

hereme888 wrote: post #266

Hmmm... GLM insists it's Gemini. https://github.com/zai-org/GLM-5/issues/79

It's a surprisingly common misconception that models contain any metadata at all about themselves in their weights. If you ask them, "What model are you?" they either retrieve the answer from the system prompt, or they hallucinate an answer. Same goes for questions about knowledge cut-off, how many parameters they have, the source of their training data, etc.
Post reply on HN