Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

z.ai

Post reply on HN
Print view View original

341–350 of 540 posts

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#341

Post by izucken »

dust42 wrote: post #233
Aurornis wrote: post #181

The benchmarks are impressive, but it's comparing to last generation models (Opus 4.5 and GPT-5.2). The competitor models are new, but they would have easily had enough time to re-run the benchmarks and update the press release by now. Although it doesn't really matter much. All of the open weights models lately come with impressive benchmarks but then don't perform as well as expected in actual use. There's clearly…

They are all just token generators without any intelligence. There is so little difference nowadays that I think in a blind test nobody will be able to differentiate the models - whether open source or closed source. Today's meme was this question: "The car wash is only 50 meters from my house. I want to get my car washed, should I drive there or walk?" Here is Claude's answer just right now: "Walk! At only 50 meters…

Tsk tsk thats adversarial prompting. Hostile even. And thats goalposting which is totally not cool, man.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#342

Post by hmottestad »

viraptor wrote: post #323
solarized wrote: post #315

Earlier quoted context omitted.

This Pelican benchmark has become irrelevant. SVG is already ubiquitous. We need a new, authentic scenario.

Like identifying names of skateboard tricks from the description? https://skatebench.t3.gg/

o3-pro is better than 5.2 pro! And GPT 5 high is best. Really quite interesting.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#343

Post by bonoboTP »

RohMin wrote: post #327
bonoboTP wrote: post #242

Earlier quoted context omitted.

It's unclear where the car is currently from your phrasing. If you add that the car is in your garage, it says you'll need to drive to get the car into the wash.

Do you think the average person would need this sort of clarification? How many of us would have recommended to walk?

Do you think this is a fundamentally unbridge-able limitation of LLMs? Do you know where we were just a year ago? Can you imagine that this will get better with upcoming releases? It's like when Gary Marcus was confidently stating that AI (at least current paradigm) will never be able to generate an image of a horse riding an astronaut. (Or full wineglasses or arbitrary clocks).

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#344

Post by throw101010 »

Havoc wrote: post #324
mark_l_watson wrote: post #288

Earlier quoted context omitted.

US Secretary of State Bressent just publicly said that the US needs to get along and cooperate with China. His tone was so different than previously in the last year that I listened to the video clip twice. Obviously for the average US tax payer getting along with China is in our interests - not so much our economic elites. I use both Chinese and US models, and Mistral in Proton’s private chat. I think it makes sense…

>His tone was so different than previously in the last year that I listened to the video clip twice. US bluff got called. A year back it looked like US held all the cards and could squeeze others without negative consequences. i.e. have cake and eat it too Since then: China has not backed down, Europe is talking de-dollarization, BRICS is starting to find a new gear on separate financial system, merciless mocking acr…

Who could have predicted that cooperation with decades old allies would be more fruitful than spitting in their faces and threatening them on a weekly basis both economically and militarily... really nobody /s

And yes, the consequence is strengthening the actual enemies of the USA, their AI progress is just one symptom of this disastrous US administration and the incompetence of Donald Trump. He really is the worst President of the USA ever, even if you were to just judge him on his leadership regarding technology... and I'm saying this while he is giving a speech about his "clean beautiful coal" right now in the White House.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#345

Post by qwytw »

dust42 wrote: post #233
Aurornis wrote: post #181

The benchmarks are impressive, but it's comparing to last generation models (Opus 4.5 and GPT-5.2). The competitor models are new, but they would have easily had enough time to re-run the benchmarks and update the press release by now. Although it doesn't really matter much. All of the open weights models lately come with impressive benchmarks but then don't perform as well as expected in actual use. There's clearly…

They are all just token generators without any intelligence. There is so little difference nowadays that I think in a blind test nobody will be able to differentiate the models - whether open source or closed source. Today's meme was this question: "The car wash is only 50 meters from my house. I want to get my car washed, should I drive there or walk?" Here is Claude's answer just right now: "Walk! At only 50 meters…

Some snippets from Kimi's 2.5 answer:

"" [...] Since you need to get your car washed, you have to bring the car to the car wash—walking there without the vehicle won't accomplish your goal [...] If it's a self-service wash, you could theoretically push the car 50 meters if it's safe and flat (unusual, but possible) [..] Consider whether you really need that specific car wash, or if a mobile detailing service might come to you [...] """

Which seems slightly (unintentionally) funny.

But to be fair all the Gemini (including flash) and GPT models I tried did understand the quesiton.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#346

Post by stevage »

viraptor wrote: post #323
solarized wrote: post #315

Earlier quoted context omitted.

This Pelican benchmark has become irrelevant. SVG is already ubiquitous. We need a new, authentic scenario.

Like identifying names of skateboard tricks from the description? https://skatebench.t3.gg/

I couldn't find an about page or similar?

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#347

Post by esafak »

esafak wrote: post #162

I got fed up with GLM-4.7 after using it for a few weeks; it was slow through z.ai and not as good as the benchmarks lead me to believe (esp. with regards to instruction following) but I'm willing to give it another try.

I forgot to mention that GLM 4.7 loves to perform destructive operations; it'll happily git reset and push to main. Put it on a very tight leash.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#348

Post by odie5533 »

valvar wrote: post #191
esafak wrote: post #162

I got fed up with GLM-4.7 after using it for a few weeks; it was slow through z.ai and not as good as the benchmarks lead me to believe (esp. with regards to instruction following) but I'm willing to give it another try.

Try Cerberas

I spent $10 in 2 minutes with that and gave up

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#349

Post by btown »

gabiruh wrote: post #301
btown wrote: post #227

Earlier quoted context omitted.

Thank you for continuing to maintain the only benchmarking system that matters! Context for the unaware: https://simonwillison.net/tags/pelican-riding-a-bicycle/

It's interesting how some features, such as green grass, a blue sky, clouds, and the sun, are ubiquitous among all of these models' responses.

If you were a pelican, wouldn't you want to go cycling on a sunny day?

Do electric pelicans dream of touching electric grass?

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#350

Post by hamdingers »

ionelaipatioaei wrote: post #228
Aurornis wrote: post #181

The benchmarks are impressive, but it's comparing to last generation models (Opus 4.5 and GPT-5.2). The competitor models are new, but they would have easily had enough time to re-run the benchmarks and update the press release by now. Although it doesn't really matter much. All of the open weights models lately come with impressive benchmarks but then don't perform as well as expected in actual use. There's clearly…

I think the only advantage that closed models have are the tools around them (claude code and codex). At this point if forced I could totally live with open models only if needed.

GLM works wonderfully with Claude, just have to set some environment variables and you're off to the races.
Post reply on HN