Addition is All You Need for Energy-efficient Language Models
1–3 of 3 posts
Re: Addition is All You Need for Energy-efficient Language Models
#2[deleted]
Re: Addition is All You Need for Energy-efficient Language Models
#3Fantastic result, on par with another similar effort: https://arxiv.org/pdf/2406.02528
It seems to me that we've stumbled upon this method of GPU-heavy matrix-multiplications in deep neural nets, and have only scratched the surface of alternative methods that are actually optimized for current CPU architectures such as Tsetlin Machines, Hyperdimensional Vectors, etc.