GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

Post by simonw » Thu, Feb 12, 2026, 12:22 AM UTC

RC_ITR wrote: post #340

simonw wrote: post #215

Pelican generated via OpenRouter: https://gist.github.com/simonw/cc4ca7815ae82562e89a9fdd99f07... Solid bird, not a great bicycle frame.

The bird not having wings, but all of us calling it a 'solid bird' is one of the most telling examples of the AI expectations gap yet. We even see its own reasoning say it needs 'webbed feet' which are nowhere to be found in the image. This pattern of considering 90% accuracy (like the level we've seemingly we've stalled out on for the MMLU and AIME) to be 'solved' is really concerning for me. AGI has to be 100% righ…

Post by icedchai » Thu, Feb 12, 2026, 12:42 AM UTC

Balinares wrote: post #313

Aurornis wrote: post #253

Earlier quoted context omitted.

If you're asking simple riddles, you shouldn't be paying for SOTA frontier models with long context. This is a silly test for the big coding models. This is like saying "all calculators are the same, nobody needs a TI-89!" and then adding 1+2 on a pocket calculator to prove your point.

I find it's a great test, actually. There are lots of "should I take the car" decisions in putting together software that's supposed to do things, and with poor judgement in how the things should be done, you typically end up with the software equivalent of a Rube-Goldberg machine that harnesses elephants to your car and uses mice to scare the elephants toward the car wash while you walk. After all, it's a short dist…

Post by TZubiri » Thu, Feb 12, 2026, 12:49 AM UTC

jaccola wrote: post #54

NiloCK wrote: post #29

Grey market fast-follow via distillation seems like an inevitable feature of the near to medium future. I've previously doubted that the N-1 or N-2 open weight models will ever be attractive to end users, especially power users. But it now seems that user preferences will be yet another saturated benchmark, that even the N-2 models will fully satisfy. Heck, even my own preferences may be getting saturated already. Op…

I have no idea how an LLM company can make any argument that their use of content to train the models is allowed that doesn't equally apply to the distillers using an LLM output. "The distilled LLM isn't stealing the content from the 'parent' LLM, it is learning from the content just as a human would, surely that can't be illegal!"...

Post by segmondy » Thu, Feb 12, 2026, 1:01 AM UTC

btown wrote: post #227

simonw wrote: post #215

Pelican generated via OpenRouter: https://gist.github.com/simonw/cc4ca7815ae82562e89a9fdd99f07... Solid bird, not a great bicycle frame.

Thank you for continuing to maintain the only benchmarking system that matters! Context for the unaware: https://simonwillison.net/tags/pelican-riding-a-bicycle/

Post by mythz » Thu, Feb 12, 2026, 1:03 AM UTC

Gracana wrote: post #319

mythz wrote: post #113

Earlier quoted context omitted.

> Doing inference with a Mac Mini to save money is more or less holding it wrong. No one's running these large models on a Mac Mini. > Of course if you buy some overpriced Apple hardware it’s going to take years to break even. Great, where can I find cheaper hardware that can run GLM 5's 745B or Kimi K2.5 1T models? Currently it requires 2x M3 Ultras (1TB VRAM) to run Kimi K2.5 at 24 tok/s [1] What are the better val…

Six months ago I'd have said EPYC Turin. You could do a heck of a build with 12Ch DDR5-6400 and a GPU or two for the dense model parts. 20k would have been a huge budget for a homelab CPU/GPU inference rig at the time. Now 20k won't buy you the memory.