SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
machinelearning.apple.com
SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
1–10 of 39 posts
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#2I think it would be better to just link directly to the paper [1]. It is a work by researchers at Apple and Meta.
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#3all this and they can't launch Apple intelligence on schedule :(
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#4all this and they can't launch Apple intelligence on schedule :(
Honestly, this seems like enabling work. Even the iPhone 16 Pro Max only seems to have 8GB RAM. If Apple Intelligence plans to do anything useful on-device, they need work like this.
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#5It sounds like they basically find part of a pseudo-random sequence that is closest to the desired data, then store the random seed and corrections (which are small so take less space).
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#6[flagged]
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#7It sounds like they basically find part of a pseudo-random sequence that is closest to the desired data, then store the random seed and corrections (which are small so take less space).
Pretty fascinating from an information theory point of view. Surprising that it works at all. Is this, like, the JPEG of uniformly distributed, uncorrelated data?
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#8Very interesting trick, using a dictionary of basis vectors which are quickly computed from a seed without storage. But the result is the same 3 or 4 bit quantization, with only a slight improvement. Their tiles are small, just 8 or 12 weights, it's why compression doesn't go too far. It would have been great if this trick lowered quantization Congrats to Apple and Meta, makes sense they did the research, this will go towards efficient serving of LLMs on phones. And it's very easy to implement.
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#9It sounds like they basically find part of a pseudo-random sequence that is closest to the desired data, then store the random seed and corrections (which are small so take less space).
Pretty fascinating from an information theory point of view. Surprising that it works at all. Is this, like, the JPEG of uniformly distributed, uncorrelated data?
You might find The Library of Babel fascinating [1, 2]
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#10Very interesting trick, using a dictionary of basis vectors which are quickly computed from a seed without storage. But the result is the same 3 or 4 bit quantization, with only a slight improvement. Their tiles are small, just 8 or 12 weights, it's why compression doesn't go too far. It would have been great if this trick lowered quantization Congrats to Apple and Meta, makes sense they did the research, this will g…
I was about to post something similar. While the research is interesting, it doesn’t offer any advantages over 3- or 4-bit quantization. I also have to assume they explored using longer tiles but found it to be ineffective — which would make sense to me from an information theory perspective.