One of the bench maxed models . Every time I tried it , it’s not on par even with other open source models .
GLM-5.1: Towards Long-Horizon Tasks
81–90 of 285 posts
Re: GLM-5.1: Towards Long-Horizon Tasks
#82How does it compare to Kimi 2.5 or Qwen 3.6 Plus?
The blog post has a benchmark comparison table with these two in it
Thanks, I missed that. It's very interesting. They're quite close, but I found Qwen 3.6 plus was just marginally better than Kimi 2.5. But looking at the stats I'll definitely give GLM 5.1 a try now. [edit: even though looking at it, it's not cheap and has a much smaller context size.And I can't tell about tool use.]
Re: GLM-5.1: Towards Long-Horizon Tasks
#83Earlier quoted context omitted.
I'm on their pro plan and I respectfully disagree - it's genuinely excellent with GLM 5.1 so long as you remember to /compact once it hits around 100k tokens. At that point it's pretty much broken and entirely unusable, but if you keep context under about 100k it's genuinely on par with Opus for me, and in some ways it's arguably better.
100k tokens it's basically nothing these days. Claude Opus 4.6M with 1M context windows is just a different ball game
The Dumb Zone for Opus has always started at 80-100k tokens. The 1M token window just made the dumb zone bigger. Probably fine if the work isn't complicated but really I never want an Opus session to go much beyond 100k.
Re: GLM-5.1: Towards Long-Horizon Tasks
#84I find the "8 hour Linux Desktop" bit disingenuous, in the fine print it's a browser page: > "build a Linux-style desktop environment as a web application" They claim "50 applications from scratch", but "Browser" and a bunch of the other apps are likely all elements. We all know that building a spec-compliant browser alone is a herculean task.
In my opinion it would be way cooler if it actually created a real Linux desktop environment instead of only a replica.
Would it succeed? Probably not, but it would be way more interesting, even if it didn't work.
I find things like Claude's C compiler way more interesting where, even though CCC is objectively bad (code is messy, generates very bad unoptimized code, etc) it at least is something cool and shows that with some human guideance it could generate something even better.
Re: GLM-5.1: Towards Long-Horizon Tasks
#85I'm crossing my fingers they release a flash version of this. GLM 4.7 Flash is the main model I use locally for agentic coding work, it's pretty incredible. Didn't find anything in the release about it - but hoping it's on the horizon.
Re: GLM-5.1: Towards Long-Horizon Tasks
#86I am on their "Coding Lite" plan, which I got a lot of use out of for a few months, but it has been seriously gimped now. Obvious quantization issues, going in circles, flipping from X to !X, injecting chinese characters. It is useless now for any serious coding work.
I am on the mid tier Coding plan to trying it out for the sake of curiosity. During off peak hour a simple 3 line CSS change took over 50 minutes and it routinely times out mid-tool and leaves dangling XML and tool uses everywhere, overwriting files badly or patching duplicate lines into files
Off peak for China or US
Re: GLM-5.1: Towards Long-Horizon Tasks
#87Earlier quoted context omitted.
I am on the mid tier Coding plan to trying it out for the sake of curiosity. During off peak hour a simple 3 line CSS change took over 50 minutes and it routinely times out mid-tool and leaves dangling XML and tool uses everywhere, overwriting files badly or patching duplicate lines into files
Off peak for China or US
Off peak for China. Off peak times are only in one timezone
Re: GLM-5.1: Towards Long-Horizon Tasks
#88Comments here seem to be talking like they've used this model for longer than a few hours -- is this true, or are y'all just sharing your initial thoughts?
My local tennis court's reservation website was broken and I couldn't cancel a reservation, and I asked GLM-5.1 if it can figure out the API. Five minutes later, I check and it had found a /cancel.php URL that accepted an ID but the ID wasn't exposed anywhere, so it found and was exploiting a blind SQL injection vulnerability to find my reservation ID. Overeager, but I was really really impressed.
Unfathomably based.
Re: GLM-5.1: Towards Long-Horizon Tasks
#89Unsloth quantizations are available on release as well. [0] The IQ4_XS is a massive 361 GB with the 754B parameters. This is definitely a model your average local LLM enthusiast is not going to be able to run even with high end hardware. [0] https://huggingface.co/unsloth/GLM-5.1-GGUF
SSD offload is always a possibility with good software support. Of course you might easily object that the model would not be "running" then, more like crawling. Still you'd be able to execute it locally and get it to respond after some time. Meanwhile we're even seeing emerging 'engram' and 'inner-layer embedding parameters' techniques where the possibility of SSD offload is planned for in advance when developing th…
For conversational purposes that may be too slow, but as a coding assistant this should work, especially if many tasks are batched, so that they may progress simultaneously through a single pass over the SSD data.
Re: GLM-5.1: Towards Long-Horizon Tasks
#90To be honest I am a bit sad as, glm5.1 is producing mich better typescript than opus or codex imo, but no matter what it does sometimes go into shizo mode at some point over longer contexts. Not always tho I have had multiple session go over 200k and be fine.
I just set the context window to 100k and manage it actively (e.g. I compact it regularly or make it write out documentation of its current state and start a new session).
For me, Opus 4.6 isn't working quite right currently, and I often use GLM 5.1 instead. I'd prefer to use peak Opus over GLM 5.1, but GLM 5.1 is an adequate fallback. It's incredible how good open-weight models have gotten.