Lossless model compression experiment: GLM-5.2 in 25% less memory
brianbell-x.github.io
Lossless model compression experiment: GLM-5.2 in 25% less memory
1–2 of 2 posts
Re: Lossless model compression experiment: GLM-5.2 in 25% less memory
#2So, if I'm understanding correctly, this is just for lower bandwidth transfers of full BF16 weights over the wire, not for serving, correct? Have you benchmarked the performance against a SOTA general purpose compression algorithm like zstd?
Also, are all that many people handling the BF16 weights directly? GLM-5.2's reference deployment is FP8, and many vendors are even serving at NVFP4 which seems to offer negligible degradation over the FP8 reference deployments.