Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.2 is the new leading open weights model on Artificial Analysis

artificialanalysis.ai

Post reply on HN
Print view View original

341–350 of 476 posts

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#341

Post by coder543 »

hereme888 wrote: post #266

Hmmm... GLM insists it's Gemini. https://github.com/zai-org/GLM-5/issues/79

Claude Sonnet 4.6 identified itself as DeepSeek repeatedly: https://www.reddit.com/r/DeepSeek/comments/1rd5jw7/claude_so...

I tested this myself a few months ago, and confirmed that it was really happening.

LLMs don't know who they are unless the system prompt tells them, and as all of them are trained on model responses that exist on the web that end up being scraped, the weights may predict a certain incorrect response. LLMs have no ability to introspect, and do not know anything about themselves, so they will hallucinate in response to that question unless they are carefully trained on that exact, pointless question.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#342

Post by x3cca »

simonw wrote: post #110

I was surprised that GLM 5.1/5.2 are not vision models - they are text input only. That's actually pretty uncommon these days. All of the OpenAI/Anthropic/Gemini models accept images, and so do the other leading open weight families - Gemma 4, Qwen 3.6, Kimi 2.x. In GLM's case image input would be useful because it's a model that scores very highly for tasks like web design, but without image input it can't take a sc…

I've been using Google ai studio as a free vision bridge. Gemma 31B is dummy capable at vision and at 1500 rpd its basically unlimited.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#343

Post by sosodev »

kristopolous wrote: post #104

I have a script that ranks these based on codingindex from Artificial Analysis. All it does is pull a json from their main table page and parses it with the fields I care about (coding). There used to be a mailing list associated with it but eh ... there wasn't much interest. I use the script every day though. Current partial output score age size name 47.1 58 large Kimi K2.6 47.5 54 large DeepSeek V4 Pro (Reasoning,…

Note that AA's coding index is only made up of two benchmarks: Terminal-Bench Hard and SciCode. I'm skeptical that it makes a good coding index. It ranks Gemma 4 31B above Deepseek V4 Flash. Having used both of those models for a broad variety of coding tasks I would choose Deepseek every day.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#344

Post by SwellJoe »

I added it to my benchmark based on Mythos-reported bugs, and it's better than GLM 5.1, but still behind several other models, maybe most directly comparable to Qwen 3.7 Max. But, several other open models, including small self-hostable ones (Gemma 4 and Qwen 3.6), found the same number of bugs, 3 of 9. Though it also gets partial credit for reporting one bug in the right spot, but kinda misunderstanding the bug. I also added Kimi K2.7-code in the same run, and it did poorly, consistent with 2.6 performance. Anyway, there are better, cheaper, models on this particular benchmark.

https://swelljoe.com/post/will-it-mythos/

(This small benchmark doesn't prove anything. It's a limited data set and each model only gets one shot at each file in the corpus. But, I find it useful for quickly sussing out if a model can reason about pretty complicated problems in code.)

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#345

Post by mrngld »

alecco wrote: post #114
cedws wrote: post #28

Earlier quoted context omitted.

OpenRouter should be penalising or banning for this.

Would that align with their VC-backed incentives?

If your users can't trust your product then I'd say that'd be a pretty strong incentive?

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#347

Post by wunderlotus »

tcp_handshaker wrote: post #137
papersail wrote: post #130

Earlier quoted context omitted.

score age size name 62.0 8 - Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) 59.1 55 - GPT-5.5 (xhigh) 58.5 55 - GPT-5.5 (high) 57.2 104 - GPT-5.4 (xhigh) 56.7 20 - Claude Opus 4.8 (Adaptive Reasoning, Max Effort) 56.2 55 - GPT-5.5 (medium) 55.5 118 - Gemini 3.1 Pro Preview 53.1 132 - GPT-5.3 Codex (xhigh) 53.1 62 - Claude Opus 4.7 (Non-reasoning, High Effort) 52.5 62 - Claude Opus 4.7 (Adaptive Re…

Short comments... - GPT 5.5 consistently the best, an opinion who gets me constant downvotes here by the Anthropic Marketeer strike force... - China is going to eat the US lunch on AI - What have European universities and companies been doing? Its like if, on a parallel past/future, Nikola Tesla and Edison would have created flying Cyberpunk machines, while Europeans researchers, would be getting together to request…

> - If Zuckerberg could be fired, after spending a total of $235 billion on AI and having NOTHING to show for...should he be fired?

Yes, if the premise was true but it’s not.

https://opper.ai/ai-roundtable/questions/bbf5a4e9-204

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#348

Post by nyrikki »

vitalyan123 wrote: post #80
benjiro29 wrote: post #59

Earlier quoted context omitted.

GLM 5.2 Max = Opus 4.8 Max in thinking behavior. The thinking chain is so similar, and so is the amount of token usage on the output. If you want reasonable token usage, you need to run it GLM 5.2 at High. There is little drop in quality from Max to High (for most tasks). And it cuts token usage by 2 a 2.5x. GLM 5.2, Max is really something you only need for complex tasks. In essence, GLM 5.2 is Opus 4.8 its little b…

distillation of thinking models is not particularly effective - both "Open"AI and Misanthropic don't show you the real chain of thought, only its severely downscaled version. both do everything in their power to combat such outrageous copyright infringement, so the bulk of unethically scrapped data the Chinese have is from several generations ago.

It is quite likely that the intermediate tokens don’t have ‘semantic import’[0]

There are methods like Habitual Reasoning Distillation or Inverted Reasoning Traces [1] that can help.

While there are reasons to hide the intermediate tokens from a IP protection stand point, there is also a need to hide more effective and efficient generating that doesn’t fit the R1 claims of an aha moment that has been debunked, but is a consumer expectation.

While hidden intermediate tokens do increase the difficulty, it is not a from barrier in itself, especially as they are billed, given information about their length.

[0] https://arxiv.org/abs/2504.09762v4

[1] https://arxiv.org/abs/2603.07267

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#349

Post by SwellJoe »

segmondy wrote: post #158
XCSme wrote: post #68

Earlier quoted context omitted.

Oh, or you meant a smaller model than GLM-5.2 with similar capabilities?

Probably not. Qwen3.(5|6)-27B seems like an "accidental freak". I'm not even sure they know what they did to create that. A decent amount of the team members left after that, so unfortunately, we might not be seeing another small model that packs such a punch for a while. Hopefully the team is studying their entire training recipe for that and is able to replicate. If they are, then a 50-70B dense model might give us…

Gemma 4 is competitive with Qwen 3.6. I had vague feelings that Qwen was better at coding tasks, based on anecdotes and public benchmarks, but I've been doing some benchmarking lately, and Gemma 4 31b is consistently beating Qwen 3.6 at the really hard stuff (finding hard security bugs, vision tasks for fixing UI layout or categorizing assets, in particular..and for vision, nothing self-hostable beats Gemma 4 12b, including 31b).

I'm still hoping for a bigger Gemma 4 version, but I think they may be worried about competing with their own hosted models, since Gemma 4 is already better than a lot of Google's proprietary models that are still available in AI Studio.

But, it is a shame that Qwen probably won't be doing more open models going forward. It is really strong for its size.

Re: GLM-5.2 is the new leading open weights model on Artificial Analysis

#350

Post by muvlon »

dr_dshiv wrote: post #216
marcus_cemes wrote: post #195

Earlier quoted context omitted.

To be honest, living in Switzerland and speaking with peers, we're just exhausted by the constant AI hype. For a lot of us, the fact that Europe isn't frantically trying to scrape the entire internet and every book in existence for the next massive model isn't a bad thing. The big players are doing their thing, like with the nuclear arms race. We regulate a lot, too much a lot of the time, but sometimes that trickles…

Sir, I would suggest that if Europe fails to be economically competitive, the downstream implications on European society will produce much worse outcomes than (for instance) data transparency… Doing things with ethical intentions does not necessarily produce outcomes that are beneficial for society at large.

Well, is this mad dash for AI producing "outcomes that are beneficial for society at large" yet? So far it looks like its mostly producing a ton of negative externalities and wealth transfer to corrupt elites.

Also, no, abandoning ethics is not an option, what a ridiculous suggestion.

Post reply on HN