GLM 5.2 beats Claude in our benchmarks

Post by secrooq » Mon, Jun 29, 2026, 11:40 AM UTC

jfaat wrote: post #262

gertlabs wrote: post #172

Earlier quoted context omitted.

GLM 5.2 is a great model, but if you only want to use the best model available, it isn't there yet. Every lab releases models that memorize benchmark answers, both intentionally and unintentionally. But we consistently find that models from Chinese labs have a wider gap between public benchmarks and our evaluations, which we designed to be less vulnerable to benchmaxxing. In multi-agent coding environments, GLM 5.2 i…

> but if you only want to use the best model available, it isn't there yet I'm trying to wrap my head around exactly why so may people seem to want the best model available when it has recently become clear that most halfway decent models can write damn good code for a fraction of the price. And the frontier models get nerfed constantly so you with open weight you can get something slightly less performant but way mo…

Post by mattmcdonagh » Mon, Jun 29, 2026, 11:55 AM UTC

Post by [deleted] » Mon, Jun 29, 2026, 11:58 AM UTC

cicko wrote: post #440

kennywinker wrote: post #382

Earlier quoted context omitted.

x40hrs/week * 50 weeks = $180k Congrats, now you’re paying an engineer’s salary to make your engineer at best 20% more productive. Better to hire another engineer, or two jrs, and build up your in house talent.

except this is way more than an engineering salary. At least in Europe.

Post by r0fl » Mon, Jun 29, 2026, 12:07 PM UTC

WinstonSmith84 wrote: post #357

40four wrote: post #351

It’s hard to argue against the open weight models if your only concern is coding. Which, for many of us hackers here in this forum, it is. But I would like to point out that the overwhelming majority of people using LLMs aren’t programmers, don’t care about coding, and couldn’t even be bothered to “vibe code”. So we should consider the bias of the output of these open weight models, and what that looks like, outside…

There is no money made from these people though .. people who are using ChatGPT to plan for their next week-end or their next vacation aren't paying a $100 or $200 monthly subscription. As for non coder office workers (accountants, PMs, etc.), they use Microsoft or Google products which all integrate AI to some extent within their products - with RAG for Sharepoint to some basic AIs to generate text or automate work…

Post by Foobar8568 » Mon, Jun 29, 2026, 12:14 PM UTC

ifwinterco wrote: post #367

jfaat wrote: post #262

Earlier quoted context omitted.

> but if you only want to use the best model available, it isn't there yet I'm trying to wrap my head around exactly why so may people seem to want the best model available when it has recently become clear that most halfway decent models can write damn good code for a fraction of the price. And the frontier models get nerfed constantly so you with open weight you can get something slightly less performant but way mo…

I think people are grouping into two flows. One group is trying to get the LLM to basically one shot everything and not properly reviewing the output. Others are using the LLM to assist their human intelligence in a tight loop. If you’re doing the former you really do need the best model available because that’s still right on the edge of what LLMs can do at best, and at worst you’re just shipping pure unmaintainable…