I see a phenomenal opportunity for old phone re-use by arraying them in some dock and making them be my "home AI."
Gemma 4 on iPhone
101–110 of 267 posts
Re: Gemma 4 on iPhone
#102Impressive model, for sure. I've been running it on my Mac, now I get to have it locally in my iPhone? I need to test this. Wait, it does agent skills and mobile actions, all local to the phone? Whaaaat? (Have to check out later! Anyone have any tips yet?) I don't normally do the whole "abliterated" thing (dealignment) but after discovering https://github.com/p-e-w/heretic , I was too tempted to try it with this mode…
>there's a whole set of ethically-justifiable but rule-flagging conversations (loosely categorizable as things like "sensitive", "ethically-borderline-but-productive" or "violating sacred cows") that are now possible with this, and at a level never before possible until now. Mind giving us a few of the examples that you plan to run in your local LLM? I am curious.
I'm not sure what you're angling at but I already gave a set of questions that are ethically legitimate yet routinely censored by the public models:
https://news.ycombinator.com/item?id=47654013
Not to mention that doing what the big model makers do literally dumbs the model down.
They should at least allow something like letting you prove your age and identity to give you access to better/unaligned models, maybe even requiring a license of some sort. Because you know what? SOMEONE in there absolutely has access to the completely uncensored versions of the latest models.
Re: Gemma 4 on iPhone
#103Earlier quoted context omitted.
For me the hallucination and gaslighting is like taking a step back in time a couple of years. It even fails the “r’s in strawberry” question. How nostalgic. It’s very impressive that this can run locally. And I hope we will continue to be able to run couple-year-old-equivalent models locally going forward.
I haven't seen anybody else post it in this thread, but this is running on 8GB of RAM. It's not the full Gemma 4 32B model. It's a completely different thing from the full Gemma 4 experience if you were running the flagship model, almost to the point of being misleading. It's their E2B and E4B variants (so 2B and 4B but also quantized) https://ai.google.dev/gemma/docs/core/model_card_4#dense_mod...
The relevant constraint when running on a phone is power, not really RAM footprint. Running the tiny E2B/E4B models makes sense, this is essentially what they're designed for.
Re: Gemma 4 on iPhone
#104Most of the models are not available. I’m guessing they will become available soon enough… At least I hope.
Re: Gemma 4 on iPhone
#105Would it work locally on a Mac Pro M4 24gb? If so I'd really appreciate a step-by-step guide.
These E2B and E4B models are very small so that they can fit into phones with around 8gb of RAM. You can get away with a much larger model. Just run:
brew install ollama
ollama run gemma4:26b-a4b-it-q4_K_MRe: Gemma 4 on iPhone
#106OP Here. It is my firm belief that the only realistic use of AI in the future is either locally on-device for almost free, or in the cloud but way more expensive then it is today. The latter option will only bemusedly for tasks that humans are more expensive or much slower in. This Gemma 4 model gives me hope for a future Siri or other with iPhone and macOS integration, “Her” (as in the movie) style.
Did you really watch “Her” and think this is a future that should happen?? Seriously????
What does what they said have anything to do with Her? Local LLMs are better than big corporations owning your data and offering LLMs for a huge cost.
Re: Gemma 4 on iPhone
#107Re: Gemma 4 on iPhone
#108Earlier quoted context omitted.
> And there's a whole set of ethically-justifiable but rule-flagging conversations (loosely categorizable as things like "sensitive", "ethically-borderline-but-productive" or "violating sacred cows") that are now possible with this, and at a level never before possible until now. I checked the abliterate script and I don't yet understand what it does or what the result is. What are the conversations this enables?
1) Coming up with any valid criticism of Islam at all (for some reason, criticisms of Christianity or Judaism are perfectly allowed even with public models!). 2) Asking questions about sketchy things. Simply asking should not be censored. 3) I don't use it for this, but porn or foul language. 4) Imitating or representing a public figure is often blocked. 5) Asking security-related questions when you are trying to do…
> Coming up with any valid criticism of Islam at all (for some reason, criticisms of Christianity or Judaism are perfectly allowed even with public models!).
When’s the last time you tried this? ChatGPT and Gemini have no trouble responding with all the common criticisms of Islam.
Re: Gemma 4 on iPhone
#109Impressive model, for sure. I've been running it on my Mac, now I get to have it locally in my iPhone? I need to test this. Wait, it does agent skills and mobile actions, all local to the phone? Whaaaat? (Have to check out later! Anyone have any tips yet?) I don't normally do the whole "abliterated" thing (dealignment) but after discovering https://github.com/p-e-w/heretic , I was too tempted to try it with this mode…
I run mlx models with omlx[1] on my mac and it works really well. [1] https://github.com/jundot/omlx
Holy hell, how new is this? I've never heard of it, looks great!
Re: Gemma 4 on iPhone
#110Are these models open source? If so this is Google’s attempt to collect user data from their models.