Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.3 is now open-weight

huggingface.co

Post reply on HN
Print view View original

71–80 of 296 posts

Re: GLM-5.3 is now open-weight

#71

Post by walrus01 »

peri-cl wrote: post #68
walrus01 wrote: post #20

Earlier quoted context omitted.

One could also run it locally on a used dual xeon (or amd-equivalent) server with 512GB RAM, albeit slower, if you have a useful workflow for it that's like "take this day's efforts and run it through various analysis agents", combined with giving it one-shot tasks/modules to build overnight. You would want a place like a garage or basement to put the server because it'll be loud.

> "dual xeon" Does inference make full use of the memory bandwidth in a NUMA system?

Memory throughput performance is better than many equivalently priced (talking about the cheaper end of the used/refurb market here) single socket systems because each CPU is talking to its own banks of RAM.

Using an example of llama-server on a 256GB (each CPU is attached to 128GB) system running qwen 3.8-flash-next Q8, which takes up about 188GB with the model and context, it's not that the inference engine 'knows' about NUMA, but if you use --load-mode mlock to load the whole thing into memory at once at the time llama-server launches, the kernel will distribute it relatively evenly across the memory connected to both CPUs.

Re: GLM-5.3 is now open-weight

#72

Post by DenisM »

nkmnz wrote: post #32
johnnyApplePRNG wrote: post #7

Earlier quoted context omitted.

I'm starting to think Opus 4.8 is significantly smaller than most people assume. If it's significantly larger than GLM 5.3 (I've heard some insane guesstimates out there like upwards of 5T params or more), that would prove rather embarrassing for Anthropic.

It seems like there is tradeoff between model size and the need for tool use, which - in my mind - is quite costly in terms of time and tokens. More detailed world knowledge requires an exponential increase in model size, but most knowledge can be acquired ad hoc using search or database queries. This will fail for questions where the model lacks the knowledge to ask the right questions, but maybe this could be solve…

Or calling into a full-knowledge model “I’m facing problem x, how do I ask myself the right questions?”

I should do that myself, come think of it.

Re: GLM-5.3 is now open-weight

#73

Post by sickcodebruh »

revolvingthrow wrote: post #16

GLM 5.3 is probably the sweet spot open weights model if you want to go beyond deepseek flash or the new glm flash. I used it with pi and had a fairly good time, especially since it’s less touchy about cyber and whatnot than the US guys. It’s slightly behind Kimi in ability but it’s a lot easier to run it, I’d expect prices (and speed!) from third parties to be noticeably better. Assuming you’re willing to drop a fat…

When we consider:

* LLM usage is new for the world

* Models are evolving quickly with high worldwide competition

* Hardware is evolving despite RAM shortages

Is investing a huge sum of money in equipment for local inference a wise use of money? Or are M5 Ultra and equivalently priced local inference hardware future-proof enough to be worth it relative to how the market is evolving? Maybe it’s all a question of what you’d spend otherwise on serverless or dedicated GPU spend…

Re: GLM-5.3 is now open-weight

#74

Post by benjiro29 »

swiftcoder wrote: post #23
scosman wrote: post #8

Earlier quoted context omitted.

It's actually slightly more expensive ($0.50 vs $0.48), but there's a temporary 50% discount. I've seen dozens of conversations about it in last 24 hours, and every major inference provided added in first 24 hours. I think it's gaining plenty of traction.

It's interesting that OpenCode Go is treating it as 2x more expensive than DeepSeek Flash, even factoring in the 50% discount

OpenCode Go is probably using quantized down DS4Flash. They outsourced to 3th party providers to keep the cost down, and being able to provide that $30 value (instead of the initial $60 > $15).

We saw the same issue with GLM 5.2 when they still published publicly who the providers are on their website. Most ran FP8 but one was doing FP4, so you had this issue where one moment you had the better FP8 and another session you had the FP4 provider.

You can check the internet archive, it was in the FAQ part before they hide/removed it. So if you looked up the providers, and the published quants, yea, ...

Given that a lot of complaints are coming from people that felt OpenCode Go Flash feel like a step down compared to old OpenCode Go/DeepSeek API directly, it smells of a quantized down provider is mixed in.

Re: GLM-5.3 is now open-weight

#75

Post by rkangel »

sickcodebruh wrote: post #73
revolvingthrow wrote: post #16

GLM 5.3 is probably the sweet spot open weights model if you want to go beyond deepseek flash or the new glm flash. I used it with pi and had a fairly good time, especially since it’s less touchy about cyber and whatnot than the US guys. It’s slightly behind Kimi in ability but it’s a lot easier to run it, I’d expect prices (and speed!) from third parties to be noticeably better. Assuming you’re willing to drop a fat…

When we consider: * LLM usage is new for the world * Models are evolving quickly with high worldwide competition * Hardware is evolving despite RAM shortages Is investing a huge sum of money in equipment for local inference a wise use of money? Or are M5 Ultra and equivalently priced local inference hardware future-proof enough to be worth it relative to how the market is evolving? Maybe it’s all a question of what y…

It is absolutely not worth buying hardware to run models for purely (long term) cost reasons. For open weights models the economies of scale means the cloud beats local significantly and your payback time is like 10 years.

However there are other reasons (e.g. privacy) that might make it worth running locally for some people.

Re: GLM-5.3 is now open-weight

#76

Post by gruez »

xscott wrote: post #66
gruez wrote: post #35

Earlier quoted context omitted.

>but what is his reasoning for not publishing it now, in 2026? What's the point of publishing it when it'll likely be outclassed by gpt-oss?

Maybe I'm reading too much between the lines, but I suspect the reason is to rub his nose in the duplicity or naivety depending on how generous you're feeling. Publishing the model would be a confession that he was wrong. AI policy is being shaped somewhat by the things Sam and Dario say. So even if you're not feeling vindictive, it's probably good to keep a track record of the previous things they have said as a Bay…

If you want to dunk on sam for "it's too dangerous to publish GPT-3", you hardly need the release of gpt-3 to prove your point. All the other open model releases already provide pretty good evidence. Moreover the fact that the model wasn't release hardly points to the fact that he wanted to save face or whatever. Most AI labs don't release their old proprietary models, so the fact that gpt-3 wasn't released tells us very little.

Re: GLM-5.3 is now open-weight

#77

Post by CookieCrisp »

Der_Einzige wrote: post #65
revolvingthrow wrote: post #16

GLM 5.3 is probably the sweet spot open weights model if you want to go beyond deepseek flash or the new glm flash. I used it with pi and had a fairly good time, especially since it’s less touchy about cyber and whatnot than the US guys. It’s slightly behind Kimi in ability but it’s a lot easier to run it, I’d expect prices (and speed!) from third parties to be noticeably better. Assuming you’re willing to drop a fat…

[flagged]

You are allowed to have an opinion on her 20k bag addiction

Re: GLM-5.3 is now open-weight

#78

Post by nkmnz »

xscott wrote: post #66
gruez wrote: post #35

Earlier quoted context omitted.

>but what is his reasoning for not publishing it now, in 2026? What's the point of publishing it when it'll likely be outclassed by gpt-oss?

Maybe I'm reading too much between the lines, but I suspect the reason is to rub his nose in the duplicity or naivety depending on how generous you're feeling. Publishing the model would be a confession that he was wrong. AI policy is being shaped somewhat by the things Sam and Dario say. So even if you're not feeling vindictive, it's probably good to keep a track record of the previous things they have said as a Bay…

Thank you, I couldn't have said that in a better way.

Re: GLM-5.3 is now open-weight

#79

Post by throwdbaaway »

walrus01 wrote: post #19
scosman wrote: post #12

Earlier quoted context omitted.

You can't compare models released 6+ months apart. GLM 5.2 was same architecture as 5.3 and not nearly as good. Takes time to build frontier intelligence and distill down to smaller sizes.

It's not that GLM5.3 in full precision unquantized is any smaller, it's 141 * 5.4GB files at approx 770GB which is about the same size as 5.2.

Hold on.. the routed experts are in FP8 now? Previously they were in BF16. Nice, this shall cut my download time by half!

Re: GLM-5.3 is now open-weight

#80

Post by seizethecheese »

nkmnz wrote: post #25

I'd like to ask Sam Altman if he still thinks that it's too dangerous to publish GPT-3. I mean, no one would use it, but what is his reasoning for not publishing it now, in 2026?

There’s not such a straightforward relationship between safety and model sis.

According to the book The Thinking Game, lower quality models at that time were considered less safe, because they could be easily tricked into doing harmful stuff. In the book, Dario (of Anthropic) was the head of safety at openAI and was responsible for pushing for 10x scaling in training to make the models safer.

It does make sense, a smart model is going to be way harder to jailbreak into telling me how to synthesize anthrax (or something).

Post reply on HN