Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.1: Towards Long-Horizon Tasks

z.ai

Post reply on HN
Print view View original

51–60 of 285 posts

Re: GLM-5.1: Towards Long-Horizon Tasks

#51

Post by jauntywundrkind »

esseph wrote: post #35
jauntywundrkind wrote: post #18

Earlier quoted context omitted.

Chiming in to second this issue. It is wildly frustrating. I suspect that this isn't the model, but something that z.ai is doing with hosting it. At launch I was related to find glm-5.1 was stable even as the context window filled all the way up (~200k). Where-as glm-5, while it could still talk and think, but had forgotten the finer points of tool use to the point where it was making grevious errors as it went (burn…

> "aside from totally going mad & speaking unpuncutaed gibberish [...] I trust it enormously." The bar is very low :(

I see where you are coming from.

But I used 70m tokens yesterday on glm-5.1 (thanks glm for having good observability of your token usage unlike openai, dunno about anthropic). And got incredible beautiful results that I super trust. It's done amazing work.

This limitation feels very shady and artificial to me, and i don't love this, but I also feel like I'm working somewhat effectively within the constraints. This does put a huge damper on people running more autonomous agentic systems, unless they have Pi or other systems that can more self adaptively improve the harness.

Re: GLM-5.1: Towards Long-Horizon Tasks

#52

Post by minimaxir »

The focus on the speed of the agent generated code as a measure of model quality is unusual and interesting. I've been focusing on intentionally benchmaxxing agentic projects (e.g. "create benchmarks, get a baseline, then make the benchmarks 1.4x faster or better without cheating the benchmarks or causing any regression in output quality") and Opus 4.6 does it very well: in Rust, it can find enough low-level optimizations to make already-fast Rust code up to 6x faster while still passing all tests.

It's a fun way to quantify the real-world performance between models that's more practical and actionable.

Re: GLM-5.1: Towards Long-Horizon Tasks

#53

Post by KaoruAoiShiho »

MegagramEnjoyer wrote: post #30
alex7o wrote: post #20

To be honest I am a bit sad as, glm5.1 is producing mich better typescript than opus or codex imo, but no matter what it does sometimes go into shizo mode at some point over longer contexts. Not always tho I have had multiple session go over 200k and be fine.

Why is that sad? A free and open source model outperforming their closed source counterparts is always a win for the users

The non-awesome context window is the sad part, but I think a better harness can deal with this.

Re: GLM-5.1: Towards Long-Horizon Tasks

#54

Post by cmrdporcupine »

RickHull wrote: post #14

I am on their "Coding Lite" plan, which I got a lot of use out of for a few months, but it has been seriously gimped now. Obvious quantization issues, going in circles, flipping from X to !X, injecting chinese characters. It is useless now for any serious coding work.

Is there any advantage to their fixed payment plans at all vs just using this model via 3rd party providers via openrouter, given how relatively cheap they tend to be on a per-token basis?

Providers like DeepInfra are already giving access to 5.1 https://deepinfra.com/zai-org/GLM-5.1

$1.40 in $4.40 out $0.26 cached

/ 1M tokens

That's more expensive than other models, but not terrible, and will go down over time, and is far far cheaper than Opus or Sonnet or GPT.

I haven't had any bad luck with DeepInfra in particular with quantization or rate limiting. But I've only heard bad things about people who used z.ai directly.

Re: GLM-5.1: Towards Long-Horizon Tasks

#56

Post by bredren »

airstrike wrote: post #29
unicornfinder wrote: post #23

Earlier quoted context omitted.

I'm on their pro plan and I respectfully disagree - it's genuinely excellent with GLM 5.1 so long as you remember to /compact once it hits around 100k tokens. At that point it's pretty much broken and entirely unusable, but if you keep context under about 100k it's genuinely on par with Opus for me, and in some ways it's arguably better.

100k tokens it's basically nothing these days. Claude Opus 4.6M with 1M context windows is just a different ball game

I had thought this, but my experience initially was that performance degradation began getting noticeable not long after crossing the old 250k barrier.

So, it has been convenient to not have hard stops / allow for extra but I still try to /clear at an actual 25% of the 1M anyhow.

This is in contrast to my use of the 1M opus model this past fall over the API, which seemed to perform more steadily.

Re: GLM-5.1: Towards Long-Horizon Tasks

#57

Post by sourcecodeplz »

johnfn wrote: post #48

GLM-5.0 is the real deal as far as open source models go. In our internal benchmarks it consistently outperforms other open source models, and was on par with things like GPT-5.2. Note that we don't use it for coding - we use it for more fuzzy tasks.

Yep, haven't tried 5.1 but for my PHP coding, GLM-5 is 99% the same as Sonnet/Opus/GPT-5 levels. It is unbelievably strong for what it costs, not to mention you can run it locally.

Re: GLM-5.1: Towards Long-Horizon Tasks

#58

Post by thawab »

airstrike wrote: post #29
unicornfinder wrote: post #23

Earlier quoted context omitted.

I'm on their pro plan and I respectfully disagree - it's genuinely excellent with GLM 5.1 so long as you remember to /compact once it hits around 100k tokens. At that point it's pretty much broken and entirely unusable, but if you keep context under about 100k it's genuinely on par with Opus for me, and in some ways it's arguably better.

100k tokens it's basically nothing these days. Claude Opus 4.6M with 1M context windows is just a different ball game

Don’t want to disappoint you, but above 200k opus memory is like a gold fish. You need to be below 150k to get good research and implementation.

Re: GLM-5.1: Towards Long-Horizon Tasks

#59

Post by syntaxing »

airstrike wrote: post #29
unicornfinder wrote: post #23

Earlier quoted context omitted.

I'm on their pro plan and I respectfully disagree - it's genuinely excellent with GLM 5.1 so long as you remember to /compact once it hits around 100k tokens. At that point it's pretty much broken and entirely unusable, but if you keep context under about 100k it's genuinely on par with Opus for me, and in some ways it's arguably better.

100k tokens it's basically nothing these days. Claude Opus 4.6M with 1M context windows is just a different ball game

I’m genuinely surprised. I use copilot at work which is capped at 128K regardless of model and it’s a monorepo. Admittedly I know our code base really well so I can point towards different things quickly directly but I don’t think I ever needed compacting more than a handful in the past year. Let alone 1M tokens.

Re: GLM-5.1: Towards Long-Horizon Tasks

#60

Post by HumanOstrich »

bigyabai wrote: post #8

It's an okay model. My biggest issue using GLM 5.1 in OpenCode is that it loses coherency over longer contexts. When you crest 128k tokens, there's a high chance that the model will start spouting gibberish until you compact the history. For short-term bugfixing and tweaks though, it does about what I'd expect from Sonnet for a pretty low price.

I wonder if running the compaction in a degraded state produces a subpar summary to continue with.
Post reply on HN