[deleted]
SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
21–30 of 39 posts
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#22all this and they can't launch Apple intelligence on schedule :(
Research vs productization. Very different.
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#23What did Zuck mean that Llama 4 Behemoth is already the highest performing base model and hasnt even done training yet? What are the benchmarks then?
Does he mean they did pretraining but not fine tuning?
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#24all this and they can't launch Apple intelligence on schedule :(
This unblocks future product work. You have to lay the groundwork.
We need advancements like this if we want on-device AI to work well. This is the kind of thing Apple Silicon needs especially. It's weak relative to Nvidia consumer chips.
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#25all this and they can't launch Apple intelligence on schedule :(
Honestly, this seems like enabling work. Even the iPhone 16 Pro Max only seems to have 8GB RAM. If Apple Intelligence plans to do anything useful on-device, they need work like this.
Personally, I‘m holding off on purchasing a new iPhone for this reason, even though my 13 Pro is getting long in the tooth as a full time iOS dev. The coming generation is rumored to have more and better memory (LPDDR5X?), and cooling (vapor chamber).
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#26A variant I have been thinking of: each parameter matrix (or block) is the sum of a random matrix (generated from a seed) and a low rank matrix (a LoRA). I'd like to experiment training from scratch in that setting.
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#27A variant I have been thinking of: each parameter matrix (or block) is the sum of a random matrix (generated from a seed) and a low rank matrix (a LoRA). I'd like to experiment training from scratch in that setting.
There's a related write-up here you might find interesting: https://wandb.ai/learning-at-home/LM_OWT/reports/Parameter-s...
It covers some experiments on weight tying, one of which is actually LoRA and random weights.
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#28Earlier quoted context omitted.
For zero-shot accuracy from Table 3: * LLaMA 3 8B: baseline 72.26, 4-bit 71.31, 3-bit 62.79 * LLaMA 3 70B: baseline 79.51, 4-bit 78.06, 3-bit 74.68 These results seem comparable to modern quantization methods—for example, the ~4-bit results for smaller LLaMA models listed here: https://ai.meta.com/blog/meta-llama-quantized-lightweight-mo...
I don't see any comparable numbers on the page you linked. Seems to only have numbers for 1B and 3B parameter models. Comparisons to AWQ and OmniQuant in Table 3 seem quite favorable with SeedLM showing 10% - 50% better performance. Also seems like the techniques may be possible to combine.
As a rule of thumb, the bigger the model is, the more graciously it degrades under quantisation. So you may assume performance loss for a 8B model would be lower than for a 3B model. (I know that doesn't make up for missing numbers in link, just fyi.)
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#29I suspect an April fools joke.
In general, compression using PRNGs is not a thing. There might be a special exception for this case, but I somewhat doubt it. =)
Re: SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
#30I suspect an April fools joke. In general, compression using PRNGs is not a thing. There might be a special exception for this case, but I somewhat doubt it. =)
The version on Arxiv dates to October 2024, which likely rules this out.