Unfortunately Apple appears to be blocking the use of these llms within apps on their app store.
I've been trying to ship an app that contains local llms and have hit a brick wall with issue 2.5.2
Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
11–20 of 196 posts
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#12Related: Gemma 4 on iPhone (254 comments) - https://news.ycombinator.com/item?id=47652561
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#13Is the output coherent though? I am yet to see a local model working on consumer grade hardware being actually useful.
I run qwen3.5 122b on a Framework Desktop at 35/ts as a daily driver doing security and OS systems and software engineering. Never paid an LLM provider and I have no reason to ever start.
What spec of Framework Desktop do you run this on?
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#14Is the output coherent though? I am yet to see a local model working on consumer grade hardware being actually useful.
Qwen3.5-9b and Qwen3.5-27b are pretty coherent on my 24G android phone
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#15Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#16is there a comparison of it running on iPhone vs. Android phones?
You can run Android on just about anything so it boils down to Linux GPU benchmarks.
That doesn't answer the question, I'm curious too. I think there's a speed and battery advantage on the A19 Pro chip compared to the Snapdragon 8 Elite Gen 5 chip, but to know for sure one has to run the same model used in the most efficient way on both machines (flagships ios and android).
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#17> edge AI deployment Isn't the "edge" meant to be computing near the user, but not on their devices?
Your device is the ultimate edge. The next frontier would be running models on your wetware.
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#18For those who would like an example of its output, I'm currently working through creating a small, free (cc0, public domain) encyclopedia (just a couple of thousand entries) of core concepts in Biology and Health Sciences, Physical Sciences, and Technology. Each entry is being entirely written by Gemma 4:e4b (the 10 GB model.) I believe that this may be slightly larger than the size of the model that runs locally on phones, so perhaps this model is slightly better, but the output is similar. Here is an example entry:
Seems pretty good to me!
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#19ESET is blocking this site saying:
Threat found This web page may contain dangerous content that can provide remote access to an infected device, leak sensitive data from the device or harm the targeted device. Threat: JS/Agent.RDW trojan
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#20> edge AI deployment Isn't the "edge" meant to be computing near the user, but not on their devices?
It depends, because edge is a meaningless term and people choose what they want for it. In 2022, we set up a call with a vendor for ‘edge’ AI. Their edge meant something like 5kW, and our edge was a single raspberry pi in the best case.