Llama3 running locally on iPhone 15 Pro
1–10 of 59 posts
Re: Llama3 running locally on iPhone 15 Pro
#2[deleted]
Re: Llama3 running locally on iPhone 15 Pro
#3App live on App Store:
https://apps.apple.com/us/app/local-brain/id6476066730
Re: Llama3 running locally on iPhone 15 Pro
#4You can run it on Android too. Not very fast though.
Re: Llama3 running locally on iPhone 15 Pro
#5For 3 seconds I was hoping it might support 13 mini, but 4GB of RAM is not enough :(
Re: Llama3 running locally on iPhone 15 Pro
#6It's fascinating. I hope to see more powerful large models on various mobile devices in the future.
Re: Llama3 running locally on iPhone 15 Pro
#7For 3 seconds I was hoping it might support 13 mini, but 4GB of RAM is not enough :(
My current and previous MacBooks have had 16GB and I've been fine with it, but given local models I think I'm going to have to go to whatever will be the maximum RAM available for the next one. It runs 13b models quite well with Ollama, but I tried `mixtral-8x7b` and saw 0.25 tokens/second speeds; I suppose I should be amazed that it ran at all.
Similarly, I am for the first time going to care about how much RAM is in my next iPhone. My iPhone 13's 4GB is, as in your case, inadequate.
Re: Llama3 running locally on iPhone 15 Pro
#8This should be attempted with phi-3 when the weights are released tomorrow.
Re: Llama3 running locally on iPhone 15 Pro
#9App live on App Store: https://apps.apple.com/us/app/local-brain/id6476066730
Wonder why it has a 2 star rating
Re: Llama3 running locally on iPhone 15 Pro
#10I’m sure my iPhone 14 Pro with 6GB RAM can handle the 4GB weights no?
Says device is unsupported :(