GLM-5.2 – How to Run Locally
unsloth.ai
GLM-5.2 – How to Run Locally
1–10 of 328 posts
Re: GLM-5.2 – How to Run Locally
#2So close! My machine with 192GB RAM + RTX 3090 24GB can almost run this. It says it needs 24GB of VRAM and 256GB of RAM for MoE offloading.
https://unsloth.ai/docs/models/glm-5.2#usage-guide
In a prior thread, someone said it would take $500k in hardware:
Re: GLM-5.2 – How to Run Locally
#3So close! My machine with 192GB RAM + RTX 3090 24GB can almost run this. It says it needs 24GB of VRAM and 256GB of RAM for MoE offloading. https://unsloth.ai/docs/models/glm-5.2#usage-guide In a prior thread, someone said it would take $500k in hardware: https://news.ycombinator.com/item?id=48629970
With 2 wouldn’t have good results. Ideal range for coding is at least Q8.
Re: GLM-5.2 – How to Run Locally
#4So close! My machine with 192GB RAM + RTX 3090 24GB can almost run this. It says it needs 24GB of VRAM and 256GB of RAM for MoE offloading. https://unsloth.ai/docs/models/glm-5.2#usage-guide In a prior thread, someone said it would take $500k in hardware: https://news.ycombinator.com/item?id=48629970
With 2 wouldn’t have good results. Ideal range for coding is at least Q8.
According to this very article, 4-bit dynamic is essentially lossless
Re: GLM-5.2 – How to Run Locally
#5So close! My machine with 192GB RAM + RTX 3090 24GB can almost run this. It says it needs 24GB of VRAM and 256GB of RAM for MoE offloading. https://unsloth.ai/docs/models/glm-5.2#usage-guide In a prior thread, someone said it would take $500k in hardware: https://news.ycombinator.com/item?id=48629970
I have the RAM, but not the VRAM. What kind of speed/tps could you expect from a 3090 with 24GBs of RAM? I am somewhat tempted to pick a GPU with 24GBs of RAM.
Re: GLM-5.2 – How to Run Locally
#6wonder if AMD's new ai chip can run this with ease? I'm seriously consider buying it. GLM 5.2 is just shy of GPT 5.4 so I would welcome offloading any grunt work locally
I am very excited for local LLMs I think we may have GPT 5.5-xhigh level of performance for under 2000 EUR
This should put more pressure on the frontier models to avoid sitting on any fancy stuff and lower token prices as a whole.
Nothing beats a local LLM disconnected from the cloud.
Re: GLM-5.2 – How to Run Locally
#7I feel like the gap is closing to be able to run good enough models locally even for coding and I would assume it could make some companies a bit nervous. Am I wrong about that?
Re: GLM-5.2 – How to Run Locally
#8I feel like the gap is closing to be able to run good enough models locally even for coding and I would assume it could make some companies a bit nervous. Am I wrong about that?
The RAM requirements are still pretty painful.
Re: GLM-5.2 – How to Run Locally
#9I feel like the gap is closing to be able to run good enough models locally even for coding and I would assume it could make some companies a bit nervous. Am I wrong about that?
The hardware requirements to run this locally are still very high. Seems far enough off mainstream for those companies not to be too worried yet.
Re: GLM-5.2 – How to Run Locally
#10wonder if AMD's new ai chip can run this with ease? I'm seriously consider buying it. GLM 5.2 is just shy of GPT 5.4 so I would welcome offloading any grunt work locally I am very excited for local LLMs I think we may have GPT 5.5-xhigh level of performance for under 2000 EUR This should put more pressure on the frontier models to avoid sitting on any fancy stuff and lower token prices as a whole. Nothing beats a loc…
At full quantization GLM 5.2 may be close to GPT 5.4. But at Q2 or whatever one needs in order to run it on a pro-sumer device it will be worse.
Also I m not sure where you are getting the under 2k value. I bought a Framework desktop 128GB last year and my setup was around 2.7k. The same setup now sells for around 4.7k.