Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.3 is now open-weight

huggingface.co

Post reply on HN
Print view View original

81–90 of 298 posts

Re: GLM-5.3 is now open-weight

#81

Post by HighGoldstein »

Der_Einzige wrote: post #65
revolvingthrow wrote: post #16

GLM 5.3 is probably the sweet spot open weights model if you want to go beyond deepseek flash or the new glm flash. I used it with pi and had a fairly good time, especially since it’s less touchy about cyber and whatnot than the US guys. It’s slightly behind Kimi in ability but it’s a lot easier to run it, I’d expect prices (and speed!) from third parties to be noticeably better. Assuming you’re willing to drop a fat…

[flagged]

Not every tech worker is making top-tier US salaries. For some (I suspect not few) people on HN that $20,000 Mac is almost a year's salary.

Re: GLM-5.3 is now open-weight

#82

Post by dw_arthur »

m00dy wrote: post #5

GLM-5.3-Flash is actually cheaper than deepseek and better than deepseek but no one is talking about yet :)

Not in my experience. Tasks that would normally cost $0.08 on DSV4-Flash have cost me $0.30+ on GLM-5.3-Flash. These costs are after Deepseek's recent increase. Also GLM-5.3-Flash is so slow compared to DSV4-Flash. I would be fine with GLM-5.3-Flash if it was cheaper and at the same speed as DSV4.

I use DSV4-Flash on Max through Deepseek's API. I have been using GLM-5.3-Flash on High through Openrouter which I thought had a 50% discount. I must be doing something wrong for the costs to be off this much.

Re: GLM-5.3 is now open-weight

#83

Post by springtimesun »

lnenad wrote: post #43
revolvingthrow wrote: post #16

GLM 5.3 is probably the sweet spot open weights model if you want to go beyond deepseek flash or the new glm flash. I used it with pi and had a fairly good time, especially since it’s less touchy about cyber and whatnot than the US guys. It’s slightly behind Kimi in ability but it’s a lot easier to run it, I’d expect prices (and speed!) from third parties to be noticeably better. Assuming you’re willing to drop a fat…

I have just built an Epyc with 512gb DDR4 3200 RAM for a "reasonable" price and I'm hoping to have a setup with GLM as the architect and Qwen 27b/Next Flash as the implementer. This is 1/5 of the price of the Mac, but also probably 1/5 of the speed lol.

I’ll be very curious what you get with DDR4. I also almost went that way. I have an Epyc DDR 5 rig and the best I see is 10 tok/s. Caveat being that’s at Q8 and a 4090 doing pre fill so it could be pushed up.

The surprising thing for me is how much work you will need to cool the banks if you’re near your memory ceiling. My memory starts soft throttling at about 74C (dies may be hotter, that’s the bank temp) and will turn down speed to try to stay below 80.

Happy to send my llama.cpp config settings if you want it.

Re: GLM-5.3 is now open-weight

#84

Post by Philpax »

jchw wrote: post #70
Philpax wrote: post #53

Earlier quoted context omitted.

The fastest I was able to get my Threadripper 3960X + 2x 3090s + 256GB DDR4-3200 to run a 2-bit quant of GLM-5.2 was 8 TPS. I would expect to be in seconds-per-token territory for a pure-CPU 4-bit quant.

One thing I'd like to try is MoE offloading: I have 2x32 GiB of VRAM and 128 GiB of DDR5 running at 4800 MT/s (only 2 channels though). I've seen people post difficult to believe MoE offloading results albeit a decently long time ago with older models. Maybe there is a quant that would fit with MoE offloading? That said, I am guessing my problem is not enough RAM - but this poor consumer platform struggles to do memo…

My measurement was with MoE offloading, but there's only so much you can keep on-GPU with a 200GB quant and 48GB of VRAM. It's hard to overcome the CPU/RAM bottleneck.

For what it's worth, all of my hardware was used; I think, all-in, I'm probably at around 3k-4k USD? Not cheap, but also not the worst for something relatively versatile.

Re: GLM-5.3 is now open-weight

#86

Post by snerbles »

Philpax wrote: post #53
jchw wrote: post #51

Earlier quoted context omitted.

Honestly I suspect neither of them will be performing terribly well but with DDR4 3200 RAM I wonder if you'll be counting tokens per second or seconds per token. I mean, you do at least get a lot of memory channels at least, compared to consumer PCs. I am curious to hear what performance you get, I feel there is not enough information out there on what different setups manage to eek out.

The fastest I was able to get my Threadripper 3960X + 2x 3090s + 256GB DDR4-3200 to run a 2-bit quant of GLM-5.2 was 8 TPS. I would expect to be in seconds-per-token territory for a pure-CPU 4-bit quant.

With a 4-bit quant of GLM-5.2, I can get about 0.8-1.1 tok/s on an underclocked dual Xeon E5-2698 v4 with 512GiB of DDR4-2400. I think it was specifically a Q4_K_M quant. Of course, the time-to-first-token is absolutely atrocious.

Which is completely insane for a ten year old configuration.

Re: GLM-5.3 is now open-weight

#87

Post by ChildOfChaos »

nozzlegear wrote: post #54
ChildOfChaos wrote: post #24

How much usage do you find you get on these kinda models (I know the pricing changes a bit) compared to a $20 sub say for Google AI Pro in anti gravity? I hate how difficult it is to compare prices when looking at subscriptions. Would $20 in open router, using models like GLM get me more or less?

I think it'd get you less than a $20 sub to any of the big three. I've used it on OpenRouter and found it kind of expensive for the results, but that might change now that it's open weight and other providers can host it/compete with Z.ai. For the work I did with it, I would've rather used DeepSeek V4 Flash just because it's more economical and still gives good results IMO. Z.ai does have their own subscription, but…

Yeah I used Ox alpha earlier this week when it was free and deepseek flash when it was free on opencode. Both were great. Z.ai’s subscription doesn’t look very good versus the others.

I burn through my current Google AI pro sub for the week in about 2 and a half days so wanted something extra to add to it, but don’t want to buy any expensive ultra plan. Flash models have got me about 98% of what I need, but quotas are still a bit low.

Re: GLM-5.3 is now open-weight

#88

Post by [deleted] »

ChildOfChaos wrote: post #24

How much usage do you find you get on these kinda models (I know the pricing changes a bit) compared to a $20 sub say for Google AI Pro in anti gravity? I hate how difficult it is to compare prices when looking at subscriptions. Would $20 in open router, using models like GLM get me more or less?

[deleted]

Re: GLM-5.3 is now open-weight

#89

Post by mmastrac »

villish wrote: post #62
mmastrac wrote: post #31

I previously posted that DS4Flash was _good_ but not _great_ on two DGX Sparks, but I have to say that GLM-5.3 is pretty amazing. It's been able to tackle all the random hard problems I've thrown at it and it has the intuition that DS4Flash seems to lack. We're nowhere near a Fable-class model IMO, but things are going to get interesting in this next year.

What quant are you running and tps?

NVFP4 ~20-30tps (MTP + vision, no dflash2).

Re: GLM-5.3 is now open-weight

#90

Post by r_lee »

everforward wrote: post #30
jasonjmcghee wrote: post #15

Earlier quoted context omitted.

I hear the argument here, but isn't it possible it has dramatically more knowledge and when you get outside the common cases many of us use it for, it'll have completely different capabilities? I feel like most benchmarks cluster on a reasonably limited area of human knowledge

Sort of depends on how well the core reasoning works. It’s not a big effort to connect an LLM to a search provider. You do pay for the tokens, but in theory on a smaller model each token is cheaper.

honestly using search isn't that great, you mostly get SEO slop, it usually won't help the model ask the right questions
Post reply on HN