Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.2 is the new leading open weights model on Artificial Analysis

artificialanalysis.ai

Post reply on HN
Print view View original

81–90 of 476 posts

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#81

Post by XCSme »

XCSme wrote: post #53

In my tests[0] GLM-5.2 is not much better than GLM-5, and overall DeepSeek V4 Flash seems to be the better/more cost-effective choice: [0]: https://aibenchy.com/compare/deepseek-deepseek-v4-flash-high...

I think the problem is, as can also be seen on other benchmarks, is that most models nowadays are focused more and more purely on tool calling and coding.

This means, that models are losing more and more general and domain-specific knowledge.

Look at those graphs on ARtificialAnalysis, GLM-5.1 still performs similarly or better:

AA-Omnisicence Accuracy: https://i.snipboard.io/5DYmpx.jpg

IFBench: https://i.snipboard.io/74kg0R.jpg

I still feel like models are not getting any smarter for a few months already, they just changed their training to be focused more on some areas than others, so shifting the intelligence from one place to another, not necessarily increasing the overall intelligence or "AGI" score.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#82

Post by wongarsu »

It's also third best overall on "AA-Omniscience Non-Hallucination Rate", far higher than DeepSeek, GPT 5.5 or Fable.

That's the one benchmark that allows LLMs to answer "I don't know" and punishes them for trying to bullshit their way through the questions

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#83

Post by osti »

kissgyorgy wrote: post #72

I tried it today through Openrouter and the API is atrocious. I got multiple rate limit and random errors every turn. Somebody wrote [1]; "I am never touching Minimax or GLM again. Their APIs had constant outages and I had to restart my runs multiple times — after burning money on the runs that failed midway." and I 100% agree. The model might be good, but if the API is so bad, it's effectively useless. [1]: https://…

I indeed got a few timeouts yesterday using the official API, I imagine for the coding plan users it'll be even worse.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#84

Post by CubsFan1060 »

petesergeant wrote: post #62
CubsFan1060 wrote: post #58

Knowing very little about how to run these, how close are we to medium or larger businesses starting to buy hardware to run models like this to keep the models local? It’s expensive, and not as capable as the frontier models, but would have some pretty big benefits around privacy and agency.

Unless you have genuine national security concerns, you’d be better off just negotiating a commercial agreement with privacy protections with a couple of existing vendors.

I think that's true until it isn't, which may end up being the problem. Fable/Mythos doesn't fall under the ZDR agreements with Anthropic. And I'm curious if others will follow suit.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#85

Post by Pragmata »

XCSme wrote: post #68
XCSme wrote: post #67

Earlier quoted context omitted.

Which Opus? GLM-5.2 is already close to Opus-4.7 level: https://aibenchy.com/compare/anthropic-claude-opus-4-7-mediu...

Oh, or you meant a smaller model than GLM-5.2 with similar capabilities?

Yep! I'm running things locally on a RTX5080 + RTX1060 + 64GB DDR5 ram, and would love to get a more capable model if possible!

QWEN3.6 27b is pretty good, but i can still notice some spots where it's not as good as the frontier models.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#86

Post by duskdozer »

vitalyan123 wrote: post #80
benjiro29 wrote: post #59

Earlier quoted context omitted.

GLM 5.2 Max = Opus 4.8 Max in thinking behavior. The thinking chain is so similar, and so is the amount of token usage on the output. If you want reasonable token usage, you need to run it GLM 5.2 at High. There is little drop in quality from Max to High (for most tasks). And it cuts token usage by 2 a 2.5x. GLM 5.2, Max is really something you only need for complex tasks. In essence, GLM 5.2 is Opus 4.8 its little b…

distillation of thinking models is not particularly effective - both "Open"AI and Misanthropic don't show you the real chain of thought, only its severely downscaled version. both do everything in their power to combat such outrageous copyright infringement, so the bulk of unethically scrapped data the Chinese have is from several generations ago.

>such outrageous copyright infringement

Sarcasm, considering the source of their own training data?

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#87

Post by mikeocool »

vorticalbox wrote: post #29
Tiberium wrote: post #2

It seems to really be a nice step-up and is getting quite close to the frontier. I wish they'd start focusing on the reasoning efficiency now, though. I have a simple (relatively) test task to evaluate LLMs: writing a simple math evaluator library in Nim (it's about 400-600 lines total max), and GLM 5.2 (xhigh which maps to max effort) spent over 15 minutes (!) reasoning, spending about 45k tokens, before it finally…

This is a problem I find with opus is will spend so long thinking then going “but wait what if” To point where I stop it and simple tell it to “start writing code you can work it out as you go along” Seems writers block also effects LLM

Seriously. Whenever I read the thinking output I get mad and turn down effort to medium or low.

Just output the code and we’ll work through it!

I feel similarly about having codex review claude’s plans. I don’t think I’ve ever seen it catch a major issue. It just points out things that would have inevitably been addressed during implementation anyway.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#88

Post by wongarsu »

CubsFan1060 wrote: post #58

Knowing very little about how to run these, how close are we to medium or larger businesses starting to buy hardware to run models like this to keep the models local? It’s expensive, and not as capable as the frontier models, but would have some pretty big benefits around privacy and agency.

I know of multiple businesses in Europe that have been doing that for a while with 70B models, and are upgrading hardware to run the new crop of 700B-1T models (really started around Kimi K2, but buying and hosting that kind of hardware takes time)

Not everyone is willing (or even legally able) to send their trade secrets to OpenAI or Anthropic

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#90

Post by tancop »

petesergeant wrote: post #62
CubsFan1060 wrote: post #58

Knowing very little about how to run these, how close are we to medium or larger businesses starting to buy hardware to run models like this to keep the models local? It’s expensive, and not as capable as the frontier models, but would have some pretty big benefits around privacy and agency.

Unless you have genuine national security concerns, you’d be better off just negotiating a commercial agreement with privacy protections with a couple of existing vendors.

if you can afford the investment you get stable low costs for years with better security (at least if your cyber team is good). its even better in regulated industries where some vendors might add a premium for hipaa/soc/pci dss compliance to the point its a lot cheaper to self host. for a smaller business its not worth it and you should just use a hosted open model.
Post reply on HN