Looking at HF, it looks like the unquantized version is half the size of glm-5.2 756 GB vs 1.51 TB. I wonder how they were able to optimize it this well
FP8 experts not BF16.
GLM-5.3 is now open-weight
111–120 of 296 posts
Re: GLM-5.3 is now open-weight
#112Looking at HF, it looks like the unquantized version is half the size of glm-5.2 756 GB vs 1.51 TB. I wonder how they were able to optimize it this well
FP8 vs BF16. But yeah, nice job.
Re: GLM-5.3 is now open-weight
#113Earlier quoted context omitted.
When we consider: * LLM usage is new for the world * Models are evolving quickly with high worldwide competition * Hardware is evolving despite RAM shortages Is investing a huge sum of money in equipment for local inference a wise use of money? Or are M5 Ultra and equivalently priced local inference hardware future-proof enough to be worth it relative to how the market is evolving? Maybe it’s all a question of what y…
So far I don’t regret buying an M1 Max device with 32Gb of RAM. The models available for it keep getting better (running just about okay for interactive use) and 400 GB/s of bandwidth is still considered a lot. The models are currently improving much faster than the hardware and this doesn’t seem to have plateaued yet.
Cool! I'm thinking about a local set up. What's your usual tokens/second rate?
Re: GLM-5.3 is now open-weight
#114Earlier quoted context omitted.
One thing I'd like to try is MoE offloading: I have 2x32 GiB of VRAM and 128 GiB of DDR5 running at 4800 MT/s (only 2 channels though). I've seen people post difficult to believe MoE offloading results albeit a decently long time ago with older models. Maybe there is a quant that would fit with MoE offloading? That said, I am guessing my problem is not enough RAM - but this poor consumer platform struggles to do memo…
My measurement was with MoE offloading, but there's only so much you can keep on-GPU with a 200GB quant and 48GB of VRAM. It's hard to overcome the CPU/RAM bottleneck. For what it's worth, all of my hardware was used; I think, all-in, I'm probably at around 3k-4k USD? Not cheap, but also not the worst for something relatively versatile.
Ah, I see - so MoE offloading is no savior. A shame but no surprise either.
Re: GLM-5.3 is now open-weight
#115How much usage do you find you get on these kinda models (I know the pricing changes a bit) compared to a $20 sub say for Google AI Pro in anti gravity? I hate how difficult it is to compare prices when looking at subscriptions. Would $20 in open router, using models like GLM get me more or less?
I think it'd get you less than a $20 sub to any of the big three. I've used it on OpenRouter and found it kind of expensive for the results, but that might change now that it's open weight and other providers can host it/compete with Z.ai. For the work I did with it, I would've rather used DeepSeek V4 Flash just because it's more economical and still gives good results IMO. Z.ai does have their own subscription, but…
> but I haven't used it because their privacy policy was pretty buns last time I checked.
What did you find objectionable? I looked at it when I subscribed almost a year ago and I was fine with it (e.g. they don't train on your API inputs).
Re: GLM-5.3 is now open-weight
#116Earlier quoted context omitted.
I have just built an Epyc with 512gb DDR4 3200 RAM for a "reasonable" price and I'm hoping to have a setup with GLM as the architect and Qwen 27b/Next Flash as the implementer. This is 1/5 of the price of the Mac, but also probably 1/5 of the speed lol.
I’ll be very curious what you get with DDR4. I also almost went that way. I have an Epyc DDR 5 rig and the best I see is 10 tok/s. Caveat being that’s at Q8 and a 4090 doing pre fill so it could be pushed up. The surprising thing for me is how much work you will need to cool the banks if you’re near your memory ceiling. My memory starts soft throttling at about 74C (dies may be hotter, that’s the bank temp) and will…
I was running one of the older llamas (3.1 I think?) at slow-ish (10-20 tok/sec at Q4?) but OK speeds on 12 year old DDR3 ECC Xeon machine
Re: GLM-5.3 is now open-weight
#117Earlier quoted context omitted.
>but what is his reasoning for not publishing it now, in 2026? What's the point of publishing it when it'll likely be outclassed by gpt-oss?
Maybe I'm reading too much between the lines, but I suspect the reason is to rub his nose in the duplicity or naivety depending on how generous you're feeling. Publishing the model would be a confession that he was wrong. AI policy is being shaped somewhat by the things Sam and Dario say. So even if you're not feeling vindictive, it's probably good to keep a track record of the previous things they have said as a Bay…
Wasn't the whole point of not releasing because they wanted to normalize the process/concerns for future actually dangerous releases?
I think the release of kimi k3 is definitely arguably dangerous, we're already seeing consequences of elite-tier cyberoffense capabilities.
Re: GLM-5.3 is now open-weight
#118Is it possible to fine tune this model and unlock / extend its cybersecurity capabilities? I'm scared that maybe we are not ready for an open-weight model with high cybersecurity skills.
kimi already exists and yes it is more than possible. it's coming
Re: GLM-5.3 is now open-weight
#119I previously posted that DS4Flash was _good_ but not _great_ on two DGX Sparks, but I have to say that GLM-5.3 is pretty amazing. It's been able to tackle all the random hard problems I've thrown at it and it has the intuition that DS4Flash seems to lack. We're nowhere near a Fable-class model IMO, but things are going to get interesting in this next year.
I assume that's about 5.3 Flash, not full?
Re: GLM-5.3 is now open-weight
#120GLM 5.3 is probably the sweet spot open weights model if you want to go beyond deepseek flash or the new glm flash. I used it with pi and had a fairly good time, especially since it’s less touchy about cyber and whatnot than the US guys. It’s slightly behind Kimi in ability but it’s a lot easier to run it, I’d expect prices (and speed!) from third parties to be noticeably better. Assuming you’re willing to drop a fat…
When we consider: * LLM usage is new for the world * Models are evolving quickly with high worldwide competition * Hardware is evolving despite RAM shortages Is investing a huge sum of money in equipment for local inference a wise use of money? Or are M5 Ultra and equivalently priced local inference hardware future-proof enough to be worth it relative to how the market is evolving? Maybe it’s all a question of what y…
Tools vs services in my mind. There is no guarantee any provider will continue to do what they are doing for you at the price they are doing it.
The object permanence of not having to reinvent the world every time a model gets sunsetted has value.