Which app is this?
Does anything similar exist for Android?
Llama3 running locally on iPhone 15 Pro
11–20 of 59 posts
Re: Llama3 running locally on iPhone 15 Pro
#12Seems like it is the 3bit quantized version? (Judging by the file size)
And does anyone know how many tokens per second it can run?
Re: Llama3 running locally on iPhone 15 Pro
#13what is with the BS answer lol
Re: Llama3 running locally on iPhone 15 Pro
#14For 3 seconds I was hoping it might support 13 mini, but 4GB of RAM is not enough :(
No luck on 14 pro max either
Re: Llama3 running locally on iPhone 15 Pro
#15App live on App Store: https://apps.apple.com/us/app/local-brain/id6476066730
Wonder why it has a 2 star rating
On the most recent iPhone pro I have a query running (about ~15 minutes so far) and the results are really good, just really slow, but I imagine the performance is worse on an older device
Re: Llama3 running locally on iPhone 15 Pro
#16what is with the BS answer lol
Re: Llama3 running locally on iPhone 15 Pro
#17Anybody else running into issues downloading models?
Re: Llama3 running locally on iPhone 15 Pro
#18For 3 seconds I was hoping it might support 13 mini, but 4GB of RAM is not enough :(
My current and previous MacBooks have had 16GB and I've been fine with it, but given local models I think I'm going to have to go to whatever will be the maximum RAM available for the next one. It runs 13b models quite well with Ollama, but I tried `mixtral-8x7b` and saw 0.25 tokens/second speeds; I suppose I should be amazed that it ran at all. Similarly, I am for the first time going to care about how much RAM is i…
I recently upgraded from my M1 Air specifically because I had purchased it with 8gb -- silly me. Now I have 24gb, and if the Air line had more available I would have sprung for 32, or even 64gb. But I'm not paying for a faster processor just to get more memory :-/
Re: Llama3 running locally on iPhone 15 Pro
#19Which app is this? Does anything similar exist for Android?
On Android you can simply run vanilla llama.cpp inside a terminal, or indeed any stack that you would run on a Linux desktop that doesn't involve a native GUI.
Re: Llama3 running locally on iPhone 15 Pro
#20This is quite impressive to be honest.
The chat is answering at a speed of one word per few/several seconds. But still, this a nice feat.
Example recording for the curious: https://www.youtube.com/watch?v=nZEvUj-QTrI