Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.3 Artificial Analysis Benchmarks

artificialanalysis.ai

Post reply on HN
Print view View original

31–40 of 64 posts

Re: GLM-5.3 Artificial Analysis Benchmarks

#31

Post by glub »

sourcecodeplz wrote: post #16
scotttrinh wrote: post #9

I like to compare models with a similar score on cost per task and output tokens per task since those measure two things I'm interested in: cost efficiency and token efficiency. Here's how GLM-5.3 compares to other models in a similar score and against GLM-5.2 to save a few clicks for others who care about these metrics: Model Score Cost / Task Output Tokens / Task ----------------------------------------------------…

Muse Spark has a nice balance. not to mentions the Contribs version is old deepseek flash prices.

Tested muse spark 1.2 because it was rated so high on design arena, and I've missed a model that can do nice UI in the hands of an operator with no UI skills.

It produced worse UI mockups than GPT and GPT models are already the bottom of the barrel here. The only model that performed well was Kimi K3 - insanely good, but expensive.

It's hard to trust benchmarks these days.

Re: GLM-5.3 Artificial Analysis Benchmarks

#32

Post by robertn702 »

Zaheer wrote: post #7

Is it worth using these models if I have a claude code subscription already? The appeal of lower cost is nice but I haven't gotten over the switching cost yet.

FYI, you can use your Claude subscription pricing with OpenCode via Meridian[0], which also makes it easier to try out other models when they come out. You can also use your other subscriptions in OpenCode with CLIProxyAPI[1]. The switching cost was relatively high, mostly from claude code plugins but completely worth it. I'm now mostly using GLM-5.3 and Codex models via OpenCode and barely using Claude which seemed unfathomable less than two months ago.

[0] https://github.com/rynfar/meridian

[1] https://github.com/router-for-me/CLIProxyAPI

edit: reworded for clarity

Re: GLM-5.3 Artificial Analysis Benchmarks

#33

Post by Sabinus »

oceanplexian wrote: post #25
Zaheer wrote: post #7

Is it worth using these models if I have a claude code subscription already? The appeal of lower cost is nice but I haven't gotten over the switching cost yet.

I use the $200 plan w/ Anthropic and run out of tokens half way through the week and supposedly they are progressively reducing the limits on all their subs even further. At some point I will switch, $200 buys a lot of tokens on OpenRouter.

Is the conventional wisdom that the subscription price/token is better than the API price/token not valid any more? Or is access to model diversity worth the increased per token costs?

Re: GLM-5.3 Artificial Analysis Benchmarks

#34

Post by tw1984 »

scotttrinh wrote: post #9

I like to compare models with a similar score on cost per task and output tokens per task since those measure two things I'm interested in: cost efficiency and token efficiency. Here's how GLM-5.3 compares to other models in a similar score and against GLM-5.2 to save a few clicks for others who care about these metrics: Model Score Cost / Task Output Tokens / Task ----------------------------------------------------…

this is not very useful.

for over 1 billion real world users living in China, they don't have the option of paying $1.52 per task to use Opus 5, they are banned doing that due to US politics.

Re: GLM-5.3 Artificial Analysis Benchmarks

#35

Post by tw1984 »

glub wrote: post #21

I've tested GLM 5.3 on the release day and Artificial Analysis is spot on. It's a really good model. But my main takeaway was something else. I've used closed weight models for long enough that I've forgotten how good it feels to see reasoning tokens. With GPT/Claude, you kind of hope that intent was captured well, that agent had all the information, all the tools it needed, because you won't see "hmmm it seems like…

With GPT/Claude, hiding those from users to waste their tokens is a feature, not a limitation.

Re: GLM-5.3 Artificial Analysis Benchmarks

#36

Post by ac29 »

dudeinhawaii wrote: post #29
scotttrinh wrote: post #9

I like to compare models with a similar score on cost per task and output tokens per task since those measure two things I'm interested in: cost efficiency and token efficiency. Here's how GLM-5.3 compares to other models in a similar score and against GLM-5.2 to save a few clicks for others who care about these metrics: Model Score Cost / Task Output Tokens / Task ----------------------------------------------------…

It would make reading and comparing a bit easier if the data was sorted by a dimension.

Cost per task:

  Model                        Score    Cost / Task    Output Tokens / Task
  -------------------------------------------------------------------------
  Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
  Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
  GPT-5.6 Terra (max)           56.6          $0.51                  20,838
  GPT-5.6 Sol (high)            57.3          $0.52                   7,545
  GLM-5.2 (max)                 53.0          $0.56                  32,200
  GLM-5.3 (max)                 59.5          $0.68                  41,107
  GPT-5.5 (xhigh)               56.3          $0.69                  16,893
  Grok 4.6 (high)               60.9          $0.84                  21,735
  Kimi K3 (max)                 59.7          $0.84                  25,474
  GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
  Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
  Claude Opus 5 (medium)        58.6          $0.98                  12,459
  Qwen3.8 Max                   58.1          $1.13                  38,287
  GPT-5.6 Sol (max)             60.9          $1.23                  16,879
  Claude Opus 5 (high)          61.5          $1.52                  21,353
  Claude Opus 4.8 (max)         57.3          $1.65                  33,557
Benchmark score:

  Model                        Score    Cost / Task    Output Tokens / Task
  -------------------------------------------------------------------------
  Claude Opus 5 (high)          61.5          $1.52                  21,353
  GPT-5.6 Sol (max)             60.9          $1.23                  16,879
  Grok 4.6 (high)               60.9          $0.84                  21,735
  Kimi K3 (max)                 59.7          $0.84                  25,474
  GLM-5.3 (max)                 59.5          $0.68                  41,107
  GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
  Claude Opus 5 (medium)        58.6          $0.98                  12,459
  Qwen3.8 Max                   58.1          $1.13                  38,287
  Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
  Claude Opus 4.8 (max)         57.3          $1.65                  33,557
  GPT-5.6 Sol (high)            57.3          $0.52                   7,545
  Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
  GPT-5.6 Terra (max)           56.6          $0.51                  20,838
  GPT-5.5 (xhigh)               56.3          $0.69                  16,893
  Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
  GLM-5.2 (max)                 53.0          $0.56                  32,200

Re: GLM-5.3 Artificial Analysis Benchmarks

#37

Post by Barbing »

tw1984 wrote: post #34
scotttrinh wrote: post #9

I like to compare models with a similar score on cost per task and output tokens per task since those measure two things I'm interested in: cost efficiency and token efficiency. Here's how GLM-5.3 compares to other models in a similar score and against GLM-5.2 to save a few clicks for others who care about these metrics: Model Score Cost / Task Output Tokens / Task ----------------------------------------------------…

this is not very useful. for over 1 billion real world users living in China, they don't have the option of paying $1.52 per task to use Opus 5, they are banned doing that due to US politics.

What’s the reseller situation?
Post reply on HN