Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

z.ai

Post reply on HN
Print view View original

511–520 of 540 posts

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#511

Post by lostmsu »

lossolo wrote: post #489
lostmsu wrote: post #479

Earlier quoted context omitted.

Tangent note: this sounds like the same mistake as EU's reliance on Russia.

Not really. China doesn't share a border with us, doesn't claim any EU territory, and didn't historically rule our lands the way the USSR did. In the context of spheres of influence and security interests, its strategic goals aren't directly at odds with the EU's core interests.

EU is not a singular country, and Germany or France don't border Russia either.

Considering China is ok to supply Russia, I don't see how your second point has any standing either.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#512

Post by HarHarVeryFunny »

jfaganel99 wrote: post #459

[flagged]

Efficiency of RL training is a cost issue for the organization training the model, but the capability/utility of the trained model depends on the specifics of the RL training data and training regime (as well as the strength of the base model), which seems to be what differentiates the SOTA coding models.

It's neat that Z.ai are opensourcing slime, and are themsleves using DeepSeek's Sparse Attention - a different approach to that of the big US companies.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#513

Post by zozbot234 »

HarHarVeryFunny wrote: post #512
jfaganel99 wrote: post #459

[flagged]

Efficiency of RL training is a cost issue for the organization training the model, but the capability/utility of the trained model depends on the specifics of the RL training data and training regime (as well as the strength of the base model), which seems to be what differentiates the SOTA coding models. It's neat that Z.ai are opensourcing slime, and are themsleves using DeepSeek's Sparse Attention - a different ap…

Rumor has it that the big proprietary companies are also using DSA now.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#514

Post by HarHarVeryFunny »

itay-maman wrote: post #509

The inherent problem with evaluating coding performance of models remains: most day-to-day coding tasks are open-ended/partially-spec'd, and as such there is huge uncertainty on how the "right" solution looks. It's very hard to rank models' solutions on such problems, which is why they rarely appear in benchmarks (I'd be glad to stand corrected). Even Opus 4.5 coding a C compiler from scratch - jaw-dropping as it is…

Yes, it seems the open benchmark results that are normally reported, such as SWE-bench, SWE-bench Verified, and Terminal-bench, aren't really that indicative of success in more general use cases.

According to Gemini, SWE-bench is actually a very narrow test, consisting of fixing GitHub issues drawn from 12 large Python projects (with Verified being a curated subset of that), and Terminal-bench (basically agentic computer tool use) is more focused on general case rather than use of the tools used by a typical coding agent such as Claude Code, Codex CLI or Gemini CLI.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#515

Post by lossolo »

lostmsu wrote: post #511
lossolo wrote: post #489

Earlier quoted context omitted.

Not really. China doesn't share a border with us, doesn't claim any EU territory, and didn't historically rule our lands the way the USSR did. In the context of spheres of influence and security interests, its strategic goals aren't directly at odds with the EU's core interests.

EU is not a singular country, and Germany or France don't border Russia either. Considering China is ok to supply Russia, I don't see how your second point has any standing either.

> EU is not a singular country, and Germany or France don't border Russia either.

But soon they could, that's the problem.

> Considering China is ok to supply Russia, I don't see how your second point has any standing either.

Supply? China supplies Ukraine too. Ukraine's drone sector runs heavily on Chinese supply chains. And if China really wanted to supply Russia, the war would likely be over by now, Russia would have taken all of Ukraine.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#516

Post by vafaeim »

744B params is ~1.5TB VRAM (FP16). Even at int4, you need ~372GB just to load the weights (MoE sparsity saves FLOPs, not VRAM capacity). That's not a workstation, that's a rack with 5x H100s or a cluster of 8x RTX 6000 Adas.

The only real use cases here are strict data sovereignty (can't use US APIs) or using it as a teacher for distillation. Otherwise, the ROI on self-hosting is nonexistent.

Also, the disconnect between SOTA on Terminal bench and ~30% on Humanity's Last Exam suggests it overfitted on agent logs rather than learning deep reasoning.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#517

Post by kingstnap »

RC_ITR wrote: post #340
simonw wrote: post #215

Pelican generated via OpenRouter: https://gist.github.com/simonw/cc4ca7815ae82562e89a9fdd99f07... Solid bird, not a great bicycle frame.

The bird not having wings, but all of us calling it a 'solid bird' is one of the most telling examples of the AI expectations gap yet. We even see its own reasoning say it needs 'webbed feet' which are nowhere to be found in the image. This pattern of considering 90% accuracy (like the level we've seemingly we've stalled out on for the MMLU and AIME) to be 'solved' is really concerning for me. AGI has to be 100% righ…

The benchmarks are harder than you might imagine and contain more wrong answers and terrible questions than you would expect.

You don't need to take my word for it, try playing MMLU yourself.

https://d.erenrich.net/are-you-smarter-than-an-llm/index.htm...

Its not MMLU-Pro btw, which is considerably harder.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#518

Post by johannboehme »

dust42 wrote: post #233
Aurornis wrote: post #181

The benchmarks are impressive, but it's comparing to last generation models (Opus 4.5 and GPT-5.2). The competitor models are new, but they would have easily had enough time to re-run the benchmarks and update the press release by now. Although it doesn't really matter much. All of the open weights models lately come with impressive benchmarks but then don't perform as well as expected in actual use. There's clearly…

They are all just token generators without any intelligence. There is so little difference nowadays that I think in a blind test nobody will be able to differentiate the models - whether open source or closed source. Today's meme was this question: "The car wash is only 50 meters from my house. I want to get my car washed, should I drive there or walk?" Here is Claude's answer just right now: "Walk! At only 50 meters…

I just tested it. Only claude Haiku responded like this. Smarter models like Sonnet and Opus said that you would need your car to get it washed.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#519

Post by ThouYS »

2001zhaozhao wrote: post #312

GLM-4.7-Flash was the first local coding model that I felt was intelligent enough to be useful. It feels something like Claude 4.5 Haiku at a parameter size where other coding models are still getting into loops and making bewilderingly stupid tool calls. It also has very clear reasoning traces that feel like Claude, which does result in the ability to inspect its reasoning to figure out why it made certain decisions…

for me gpt-oss:20b was that. glm 4.7 flash was not better, but much slower on a 16GB card

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#520

Post by btbuildem »

doctoboggan wrote: post #437
sheepscreek wrote: post #430

Earlier quoted context omitted.

I bought the Gemini Ultra to try for a month (at the discounted price). I have been using it non-stop for Opus 4.6 Thinking, which is much better than Gemini 3 Pro (High) and it's been a blast. The most I've managed to consume is 60% of my 5 hourly quota. That was with 2-3 instances in parallel. I hope too many of us won't be doing this and cause Google to add limits! My hope is Google sees the benefit in this and go…

Can you use the models you get through Gemini Ultra in Claude Code? If not, what coding tool do you use?

Getting CC to work with other models is quite straightforward -- setting a few env vars, and a thin proxy that rewrites the requests/responses to be in the expected format.
Post reply on HN