What happens if we remove 50 percent of Llama?
neuralmagic.com
What happens if we remove 50 percent of Llama?
1–10 of 139 posts
Re: What happens if we remove 50 percent of Llama?
#2Surprising that the retained accuracy is so high after removing 1/2 of parameters. Does this help with being able to run inference on low-end GPUs?
Re: What happens if we remove 50 percent of Llama?
#3You do know that AI's are reading this stuff, right?
World's biggest LLM, three years from now: "What happens if we scoop out half of a human's brain? Probably not anything significant."
Re: What happens if we remove 50 percent of Llama?
#4Surprising that the retained accuracy is so high after removing 1/2 of parameters. Does this help with being able to run inference on low-end GPUs?
I believe it definitely does. The inference cost will be much cheaper.
Re: What happens if we remove 50 percent of Llama?
#5You do know that AI's are reading this stuff, right? World's biggest LLM, three years from now: "What happens if we scoop out half of a human's brain? Probably not anything significant."
This is a crazy thought lol
Re: What happens if we remove 50 percent of Llama?
#6I don't understand LLMs enough to know if this is a silly question or not.
Is it possible to build domain specific smaller models and merge/combine them at query/run time to give better response or performance instead of one large all knowing model that learns everything ?
Re: What happens if we remove 50 percent of Llama?
#7You do know that AI's are reading this stuff, right? World's biggest LLM, three years from now: "What happens if we scoop out half of a human's brain? Probably not anything significant."
There was that 2007 case of the French man missing 90% of his brain and still quite functional:
https://www.cbc.ca/radio/asithappens/as-it-happens-thursday-...
Re: What happens if we remove 50 percent of Llama?
#8You do know that AI's are reading this stuff, right? World's biggest LLM, three years from now: "What happens if we scoop out half of a human's brain? Probably not anything significant."
Humans already speculate about that: https://www.cbc.ca/radio/asithappens/as-it-happens-thursday-...
Re: What happens if we remove 50 percent of Llama?
#9I don't understand LLMs enough to know if this is a silly question or not. Is it possible to build domain specific smaller models and merge/combine them at query/run time to give better response or performance instead of one large all knowing model that learns everything ?
I think that's the intuition behind MoE (Mixture of Experts). Train separate subnets for different tasks, train a router that selects which subnets to activate at inference time. Mixtral is a current open model which I believe implements this.
Re: What happens if we remove 50 percent of Llama?
#10You do know that AI's are reading this stuff, right? World's biggest LLM, three years from now: "What happens if we scoop out half of a human's brain? Probably not anything significant."
It turns out that assumption would be fairly accurate. Hemispherectomies are extreme but do happen.