Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.3 Artificial Analysis Benchmarks

artificialanalysis.ai

Post reply on HN
Print view View original

21–30 of 64 posts

Re: GLM-5.3 Artificial Analysis Benchmarks

#21

Post by glub »

I've tested GLM 5.3 on the release day and Artificial Analysis is spot on. It's a really good model.

But my main takeaway was something else. I've used closed weight models for long enough that I've forgotten how good it feels to see reasoning tokens.

With GPT/Claude, you kind of hope that intent was captured well, that agent had all the information, all the tools it needed, because you won't see "hmmm it seems like nix flake isn't available here and I shouldn't install something globally" until it slopped out millions of tokens and wasted hundreds of dollars for 8 hours. With GLM and the likes, you just stop the disease right where it begins.

Re: GLM-5.3 Artificial Analysis Benchmarks

#22

Post by kolinko »

teravor wrote: post #18
scotttrinh wrote: post #9

I like to compare models with a similar score on cost per task and output tokens per task since those measure two things I'm interested in: cost efficiency and token efficiency. Here's how GLM-5.3 compares to other models in a similar score and against GLM-5.2 to save a few clicks for others who care about these metrics: Model Score Cost / Task Output Tokens / Task ----------------------------------------------------…

these $/task figures aren't very useful in my experience. it doesn't tell you how well it did the task. generally I choose models by their intelligence and then personal preference from direct experience.

they make a ton of sense for tasks that are achievable with no prob by all models - e.g. writing scripts that do a specific thing etc.

Re: GLM-5.3 Artificial Analysis Benchmarks

#23

Post by Havoc »

glub wrote: post #21

I've tested GLM 5.3 on the release day and Artificial Analysis is spot on. It's a really good model. But my main takeaway was something else. I've used closed weight models for long enough that I've forgotten how good it feels to see reasoning tokens. With GPT/Claude, you kind of hope that intent was captured well, that agent had all the information, all the tools it needed, because you won't see "hmmm it seems like…

Yes, not necessary often but being able to stop something that is going off the rails is super useful. Especially if the root cause is prompt ambiguity - inject a clarification & it recovers

Re: GLM-5.3 Artificial Analysis Benchmarks

#24

Post by teravor »

kolinko wrote: post #22
teravor wrote: post #18

Earlier quoted context omitted.

these $/task figures aren't very useful in my experience. it doesn't tell you how well it did the task. generally I choose models by their intelligence and then personal preference from direct experience.

they make a ton of sense for tasks that are achievable with no prob by all models - e.g. writing scripts that do a specific thing etc.

    > writing scripts
you can use a Gemini model completely for free doing that. no agent. aistudio.google.com

Re: GLM-5.3 Artificial Analysis Benchmarks

#25

Post by oceanplexian »

Zaheer wrote: post #7

Is it worth using these models if I have a claude code subscription already? The appeal of lower cost is nice but I haven't gotten over the switching cost yet.

I use the $200 plan w/ Anthropic and run out of tokens half way through the week and supposedly they are progressively reducing the limits on all their subs even further.

At some point I will switch, $200 buys a lot of tokens on OpenRouter.

Re: GLM-5.3 Artificial Analysis Benchmarks

#26

Post by glub »

Zaheer wrote: post #7

Is it worth using these models if I have a claude code subscription already? The appeal of lower cost is nice but I haven't gotten over the switching cost yet.

If anything, it's going to be more expensive. Price/performance ratio isn't there yet for frontier open weight models.

But regardless, you definitely should use a harness where switching models on the fly is easy. There's a reason why Anthropic uses their own proprietary formats/conventions anywhere they can - to lock you in when inference eventually commoditizes.

Re: GLM-5.3 Artificial Analysis Benchmarks

#27

Post by glub »

colingauvin wrote: post #10
Zaheer wrote: post #7

Is it worth using these models if I have a claude code subscription already? The appeal of lower cost is nice but I haven't gotten over the switching cost yet.

At least by API usage, they aren't yet lower cost than subscriptions. Not sure about GLM's subscription plans though.

GLM subscription is better than API, but significantly worse than Codex, even when used outside peak hours.

Re: GLM-5.3 Artificial Analysis Benchmarks

#28

Post by kilroy123 »

oceanplexian wrote: post #25
Zaheer wrote: post #7

Is it worth using these models if I have a claude code subscription already? The appeal of lower cost is nice but I haven't gotten over the switching cost yet.

I use the $200 plan w/ Anthropic and run out of tokens half way through the week and supposedly they are progressively reducing the limits on all their subs even further. At some point I will switch, $200 buys a lot of tokens on OpenRouter.

Same here. I'm switching to the Codex plan. It just doesn't go very far now. Especially if you use fable at all.

Re: GLM-5.3 Artificial Analysis Benchmarks

#29

Post by dudeinhawaii »

scotttrinh wrote: post #9

I like to compare models with a similar score on cost per task and output tokens per task since those measure two things I'm interested in: cost efficiency and token efficiency. Here's how GLM-5.3 compares to other models in a similar score and against GLM-5.2 to save a few clicks for others who care about these metrics: Model Score Cost / Task Output Tokens / Task ----------------------------------------------------…

It would make reading and comparing a bit easier if the data was sorted by a dimension.

Re: GLM-5.3 Artificial Analysis Benchmarks

#30

Post by glub »

Havoc wrote: post #23
glub wrote: post #21

I've tested GLM 5.3 on the release day and Artificial Analysis is spot on. It's a really good model. But my main takeaway was something else. I've used closed weight models for long enough that I've forgotten how good it feels to see reasoning tokens. With GPT/Claude, you kind of hope that intent was captured well, that agent had all the information, all the tools it needed, because you won't see "hmmm it seems like…

Yes, not necessary often but being able to stop something that is going off the rails is super useful. Especially if the root cause is prompt ambiguity - inject a clarification & it recovers

It's also starting to go beyond reasoning and it's becoming much more problematic. Reasoning is one thing, but codex, for example now encrypts agent-to-agent messages as well, and compaction. I've no idea what subagents are instructed to do, or what they reported back in native codex.

The only thing that's keeping me is the value $200 subscription provides. If that value disappears, I see no reason why not to switch to something that isn't a black box.

Post reply on HN