SparseGPT: Language Models Can Be Accurately Pruned in One-Shot
1–10 of 66 posts
Re: SparseGPT: Language Models Can Be Accurately Pruned in One-Shot
#2Wow, these results are amazing! This might be extremely helpful in reducing the memory consumption of large models in the future.
Re: SparseGPT: Language Models Can Be Accurately Pruned in One-Shot
#3wow this could make the fabled 65 billion parameter llama sparsed and pruned runnable on a 3060
Re: SparseGPT: Language Models Can Be Accurately Pruned in One-Shot
#4It looks like by pruning by a factor of 0.5, you reduce the size of the model by 50%? In practice, what is the expected observed change in the output before and after pruning?
Re: SparseGPT: Language Models Can Be Accurately Pruned in One-Shot
#5wow this could make the fabled 65 billion parameter llama sparsed and pruned runnable on a 3060
How would this be any different from running one of the lower parameter models?
Re: SparseGPT: Language Models Can Be Accurately Pruned in One-Shot
#6wow this could make the fabled 65 billion parameter llama sparsed and pruned runnable on a 3060
How would this be any different from running one of the lower parameter models?
It says in the abstract
> at minimal loss of accuracy
Suggesting that there is a lot of redundancy in the weights.
Re: SparseGPT: Language Models Can Be Accurately Pruned in One-Shot
#7Very good result, and awesome to see such great progress happen so fast.
Quantizing/pruning/deduplicating/compressing models and embeddings is still a vast orchard of low hanging fruit.
I personally think there are still quite a few multiple-orders-of-magnitude scale opportunities to accelerate inference, and we are fortunate to have strong economic incentives aligned with the problem.
Re: SparseGPT: Language Models Can Be Accurately Pruned in One-Shot
#8The robustness with which these models can be quantized, and now trimmed makes one think if they could be easily implemented some form of analog (or optical) hardware.
Re: SparseGPT: Language Models Can Be Accurately Pruned in One-Shot
#9Once you prune your model, can you get even better performance by re-training it? I've heard theories that this is the function of sleep in brains.
Re: SparseGPT: Language Models Can Be Accurately Pruned in One-Shot
#10Very good result, and awesome to see such great progress happen so fast. Quantizing/pruning/deduplicating/compressing models and embeddings is still a vast orchard of low hanging fruit. I personally think there are still quite a few multiple-orders-of-magnitude scale opportunities to accelerate inference, and we are fortunate to have strong economic incentives aligned with the problem.
So much low hanging fruit for those willing to pick it. It's a great time to be an LLM researcher.