Transformer architecture optimized for Apple Silicon
1–10 of 342 posts
Re: Transformer architecture optimized for Apple Silicon
#2(2022)
Re: Transformer architecture optimized for Apple Silicon
#3As someone entirely at sea with the rapid pace of development in this sphere:
1. Is this a new LLM from Apple?
2. Is this a way to optimize running LLMs like Llama locally on M1 macs?
3. Something else altogether?
Re: Transformer architecture optimized for Apple Silicon
#4The race is on and ecosystems are moving fast.
Re: Transformer architecture optimized for Apple Silicon
#5Maybe apple will have a bigger effect on ai adoption than any other company.
Local inference is huge for anything that requires even a little bit of privacy.
Re: Transformer architecture optimized for Apple Silicon
#6As someone entirely at sea with the rapid pace of development in this sphere: 1. Is this a new LLM from Apple? 2. Is this a way to optimize running LLMs like Llama locally on M1 macs? 3. Something else altogether?
2. A Transformer is a core building block of LLMs.
> [T]he device spec for this reference implementation is M1 or newer chips for the Mac and A14 and newer chips for the iPhone and iPad
Re: Transformer architecture optimized for Apple Silicon
#7i'd say within 5 years apple will have optimized apple silicon and their tech, along with language model improvements, such that you will be able to get gpt-4 level performance in the iPhone 19 with inference happening entirely locally.
openai is doing great work and is serious competition, but I think many underestimate big tech. once they're properly motivated they'll catch up quick. I think we can agree that openai is a sufficient motivator.
Re: Transformer architecture optimized for Apple Silicon
#8As someone entirely at sea with the rapid pace of development in this sphere: 1. Is this a new LLM from Apple? 2. Is this a way to optimize running LLMs like Llama locally on M1 macs? 3. Something else altogether?
It's none of those things. It is tweaks of other existing code to run better on apple's hardware. This other article is far more informative than the repo: https://machinelearning.apple.com/research/neural-engine-tra...
Re: Transformer architecture optimized for Apple Silicon
#9As someone entirely at sea with the rapid pace of development in this sphere: 1. Is this a new LLM from Apple? 2. Is this a way to optimize running LLMs like Llama locally on M1 macs? 3. Something else altogether?
#2. A way to optimize running LLMs locally on Apple Silicon (including iPhones)
I am just a little better informed. As I understand it, their code improves model performance and memory consumption using PyTorch and Huggingface libraries.
Re: Transformer architecture optimized for Apple Silicon
#10Weird I just read this tweet [0] arguing Apple will be launching their own secure and private LLM that runs on device (edge compute).
https://twitter.com/LinusEkenstam/status/1638999208911949845...