Live data from Hacker News

GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

z.ai

471–480 of 540 posts

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#471
post #167
post #165

Is this a lot cheaper to run (on their service or rented GPUs) than Claude or ChatGPT?

Yes. https://z.ai/subscribe

they just removed the 50% discount today

i can imagine they're a bit tight to keep those prices viable while they also have enough compute to train their new models

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#472
There is a well-known CLI tool for JSON processing called jq. I have just asked GLM-4.7 for the name of jq's built function to convert a string to lowercase. It is called ascii_downcase() according to the manual:

https://jqlang.org/manual/#ascii_downcase-ascii_upcase

However GLM-4.7 insists that is called ascii_down().

I tried to correct it and gave the exact version number, but still, after a long internal monologue, This is its final world:

"In standard jq version 1.7, the function is named ascii_down, not ascii_downcase.

If you are receiving an error that ascii_down is not defined, please verify your version with jq --version. It is possible you are using a different binary (like gojq) or a version older than 1."

GLM-5 gives me the correct answer, ascii_downcase, but I can get this in the Chat Window. Via the API I get HTTP Status 429 - too many requests.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#473
post #340
post #215

Pelican generated via OpenRouter: https://gist.github.com/simonw/cc4ca7815ae82562e89a9fdd99f07... Solid bird, not a great bicycle frame.

The bird not having wings, but all of us calling it a 'solid bird' is one of the most telling examples of the AI expectations gap yet. We even see its own reasoning say it needs 'webbed feet' which are nowhere to be found in the image. This pattern of considering 90% accuracy (like the level we've seemingly we've stalled out on for the MMLU and AIME) to be 'solved' is really concerning for me. AGI has to be 100% righ…

This test is so far beyond AGI. Try to spit out the SVG for a pelican riding a bicycle. You are only allowed to use a simple text editor. No deleting or moving the text cursor. You have 1 minute.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#474
post #34

Lets not miss that MiniMax M2.5 [1] is also available today in their Chat UI [2]. I've got subs for both and whilst GLM is better at coding, I end up using MiniMax a lot more as my general purpose fast workhorse thanks to its speed and excellent tool calling support. [1] https://news.ycombinator.com/item?id=46974878 [2] https://agent.minimax.io

My perspective aligns with this: I used to obsess over the Best Model, which I defined as "top of benchmarks", which also meant Biggest, Slowest and Most Expensive.

Then I gave two models a Real World Task.

The "Best" model took 3x longer to complete it, and cost 10x more. [0]

Now I define Best Model as "the smallest, fastest, cheapest one that can get the job done". (Currently happy with GLM-4.7 on Cerebras, at least I would be if the unlimited plan wasn't sold out ;)

I later expanded this principle when model speed crossed into the Interactive domain. Speed is not merely a feature; a sufficient difference in speed actually produces a completely new category of usage.

[0] We recently arrived at an approximation of AGI which is "put a lossy solver in an until-done loop". For most tasks we're throwing stuff at a wall to see what sticks, and the smaller models throw faster.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#475
post #215

Pelican generated via OpenRouter: https://gist.github.com/simonw/cc4ca7815ae82562e89a9fdd99f07... Solid bird, not a great bicycle frame.

That's a bike that's ergonomically designed for pelicans. It is unreasonable to expect pelicans to ride human bikes, they have different anatomy.

The next frontier:

Draw a pelican on a bicycle ergonomically designed for pelicans.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#476

It's live on openrouter now. In my personal benchmark it's bad. So far the benchmark has been a really good indicator of instruction following and agentic behaviour in general. To those who are curious, the benchmark is just the ability of model to follow a custom tool calling format. I ask it to using coding tasks using chat.md [1] + mcps. And so far it's just not able to follow it at all. [1] https://github.com/rus…

I love the idea of chat.md. I'm developing a personal text editor with vim keybindings and paused work because I couldn't think of a good interface that felt right. This could be it. I think I'll update my editor to do something like this but with intelligent "collapsing" of extra text to reduce visual noise.

Cool! Please share your work if possible!

I couldn't decide on folding and reducing noise so I'm stuck on that front. I believe there is some elegant solution that I'm missing, hope to see your take.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#477
post #472

There is a well-known CLI tool for JSON processing called jq. I have just asked GLM-4.7 for the name of jq's built function to convert a string to lowercase. It is called ascii_downcase() according to the manual: https://jqlang.org/manual/#ascii_downcase-ascii_upcase However GLM-4.7 insists that is called ascii_down(). I tried to correct it and gave the exact version number, but still, after a long internal monologue…

Seems that I cannot use GLM-5 via the API yet, because I am on the Coding-Lite Plan, the most basic paid tier.

I have also realized that I get faster and correct answer to the ascii_downcase question (even from GLM-4.7) when I submit to open.bigmodel.cn endpoint rather than the z.ai API endpoints (using the same API key). I get a mix of Chinese and Western characters in error responses from open.bigmodel.cn though, while the z.ai endpoint does only contain Western Characters.

(Just assuming that both websites are operated by the same company).

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#478
post #29

Grey market fast-follow via distillation seems like an inevitable feature of the near to medium future. I've previously doubted that the N-1 or N-2 open weight models will ever be attractive to end users, especially power users. But it now seems that user preferences will be yet another saturated benchmark, that even the N-2 models will fully satisfy. Heck, even my own preferences may be getting saturated already. Op…

Why distill, if you can run the full model yourself... or at other inference providers.

Quantization the better approach in most cases, unless you want to for instance create hybrid models ie. distilling from here and there.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#479

Earlier quoted context omitted.

I specifically do not use the CN/SG based original provider simply because I don't want my personal data traveling across the pacific. I try to only stay on US providers. Openrouter shows you what the quantization of each provider is, so you can choose a domestic one that's FP8 if you want

Funny, living in Europe, I prefer using EU and Chinese hosts because as I don't want my data going to the US. The trust in US firms and state is completely gone.

Tangent note: this sounds like the same mistake as EU's reliance on Russia.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#480

Earlier quoted context omitted.

That's a bike that's ergonomically designed for pelicans. It is unreasonable to expect pelicans to ride human bikes, they have different anatomy.

The next frontier: Draw a pelican on a bicycle ergonomically designed for pelicans.

Thereafter: Design a bike that an actual pelican can learn to ride in real life.
Post reply on HN