There are many apps to run local LLMs on both iOS & Android
Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
21–30 of 196 posts
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#22[flagged]
That bugged me too, so I started looking at other articles - they all look AI generated to me. Whole website should be banned.
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#23Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#24> edge AI deployment Isn't the "edge" meant to be computing near the user, but not on their devices?
No it does not. This is about as “edge” as AI gets.
In a general sense, edge just means moving the computation to the user, rather than in a central cloud (although the two aren’t mutually exclusive, eg Cloudflare Workers)
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#25Earlier quoted context omitted.
I run qwen3.5 122b on a Framework Desktop at 35/ts as a daily driver doing security and OS systems and software engineering. Never paid an LLM provider and I have no reason to ever start.
What spec of Framework Desktop do you run this on?
There is only one and for this model you need the one with 128GiB RAM.
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#26[flagged]
I find it fascinating that after all this time reporters still don’t even bother to proofread content for obvious AI tells. I guess nobody really cares anymore?
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#27Is the output coherent though? I am yet to see a local model working on consumer grade hardware being actually useful.
It's highly coherent (see my other comment for an example of its text output) and yes, it's useful. I am starting to use Gemma 4:e4b as my daily driver for simple commands it definitely knows, things that are too simple to use ChatGPT for. It is also able to code through moderately difficult coding tasks. If you want to see it in action, I posted a video about it here[1] (the 10 GB one is at the 2 minute mark and the 20 GB one says hello at 5 minutes 45 seconds into the video.) You can see its speed and output on simple consumer grade hardware, in this case a Mac Mini M4 with 24 GB of RAM.
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#28> edge AI deployment Isn't the "edge" meant to be computing near the user, but not on their devices?
Your device is the ultimate edge. The next frontier would be running models on your wetware.
Not just running it on your wetware, but charging you for it.
Can't wait until AI companies go from mimicking human thought to figuring how to licensing those thoughts. ;)
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#29Related: Gemma 4 on iPhone (254 comments) - https://news.ycombinator.com/item?id=47652561
Another related submission from 22 days ago : iPhone 17 Pro Demonstrated Running a 400B LLM (+700pts, +300cmts):
https://news.ycombinator.com/item?id=47490070
Re: Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference
#30Is the output coherent though? I am yet to see a local model working on consumer grade hardware being actually useful.
Google's models work quite well on my Android phone. I haven't found a use case beyond generating shitposts, but the model does its job pretty well. It's not exactly ChatGPT, but minor things like "alter the tone of this email to make it more professional" work like a charm.
You need a relatively beefy phone to run this stuff on large amounts of text, though, and you can't have every app run it because your battery wouldn't last more than an hour.
I think the real use case for apps is more like going to be something like tiny, purpose-trained models, like the 270M models Google wants people to train and use: https://developers.googleblog.com/on-device-function-calling... With these things, you can set up somewhat intelligent situational automation without having to work out logic trees and edge cases beforehand.