Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 on iPhone

apps.apple.com

Post reply on HN
Print view View original

161–170 of 267 posts

Re: Gemma 4 on iPhone

#161

Post by rock_artist »

I really believe in the future of local models.

From app developer and user, My main concern for now is bloating devices. Until we’ll have something like Apples foundation model where multiple apps could share the same model it means we have something horrible as Electron in the sense, every app is a fully blown model (browser in the electron story) instead of reusing the model.

With desktops we have DLL hell for years. But with sandboxed apps on mobile devices it becomes a bigger issue that I guess will/should be addressed by the OS.

For my app I’ve been trying to add some logic based on large model but for bloating a simple Swift app with 2-3GB of model or even few hundred MBs feels wrong doing and conflicting with code reusability concepts.

Re: Gemma 4 on iPhone

#162

Post by modeless »

It's so ridiculous that Google made a custom SoC for their phones, touting its AI performance, even calling it Tensor, and Apple is still faster at running Google's own model.

Google really ought to shut down their phone chip team. Literally every chip from them has been a disappointment. As much as I hate to say it, sticking with Qualcomm would have been the right choice.

Re: Gemma 4 on iPhone

#163

Post by ulfw »

modeless wrote: post #162

It's so ridiculous that Google made a custom SoC for their phones, touting its AI performance, even calling it Tensor, and Apple is still faster at running Google's own model. Google really ought to shut down their phone chip team. Literally every chip from them has been a disappointment. As much as I hate to say it, sticking with Qualcomm would have been the right choice.

It runs very fast on my Qualcomm Elite Gen 5 SoC Oppo Find N6

Re: Gemma 4 on iPhone

#164

Post by allpratik »

ulfw wrote: post #163
modeless wrote: post #162

It's so ridiculous that Google made a custom SoC for their phones, touting its AI performance, even calling it Tensor, and Apple is still faster at running Google's own model. Google really ought to shut down their phone chip team. Literally every chip from them has been a disappointment. As much as I hate to say it, sticking with Qualcomm would have been the right choice.

It runs very fast on my Qualcomm Elite Gen 5 SoC Oppo Find N6

How many tokens per second? Also, does it get warm/hot?

Re: Gemma 4 on iPhone

#165

Post by nothinkjustai »

eagleinparadise wrote: post #157
nothinkjustai wrote: post #113

Earlier quoted context omitted.

It’s completely vibe coded, doesn’t even run on my Mac lol

[flagged]

Yeah definitely a skill issue downloading and running the app and it crashing on startup. Just gotta skillfully press the icon I guess.

Re: Gemma 4 on iPhone

#166

Post by bigyabai »

zozbot234 wrote: post #103
dimmke wrote: post #97

Earlier quoted context omitted.

I haven't seen anybody else post it in this thread, but this is running on 8GB of RAM. It's not the full Gemma 4 32B model. It's a completely different thing from the full Gemma 4 experience if you were running the flagship model, almost to the point of being misleading. It's their E2B and E4B variants (so 2B and 4B but also quantized) https://ai.google.dev/gemma/docs/core/model_card_4#dense_mod...

The relevant constraint when running on a phone is power, not really RAM footprint. Running the tiny E2B/E4B models makes sense, this is essentially what they're designed for.

Between the GPU, NPU and big.LITTLE cores, many phones have no fewer than 4 different power profiles they can run inference at. It's about as solved as it will get without an architectural overhaul.

Re: Gemma 4 on iPhone

#167

Post by modeless »

allpratik wrote: post #164
ulfw wrote: post #163

Earlier quoted context omitted.

It runs very fast on my Qualcomm Elite Gen 5 SoC Oppo Find N6

How many tokens per second? Also, does it get warm/hot?

If this Gemma tokenizer I found online is accurate then my Pixel 10 Pro XL is getting ~22 tok/s on Gemma 4 E2B using the NPU, vs. 40 tok/s is what people are saying the MLX version gets on iPhone.

Actually I found official performance numbers from Google saying iPhone gets 56 tok/s and Qualcomm gets 52. They don't even bother listing Tensor in their table. Maybe because it would be too embarrassing. Ouch! https://ai.google.dev/edge/litert-lm/overview

Re: Gemma 4 on iPhone

#168

Post by Auracle »

tredre3 wrote: post #72
spijdar wrote: post #31

Earlier quoted context omitted.

Realistically, a lot of people do this for porn. In my experience, though, it's necessary to do anything security related. Interestingly, the big models have fewer refusals for me when I ask e.g. "in situation, how do you exploit ?", but local models will frequently flat out refuse, unless the model has been abliterated.

From what I've seen gemma 4 doesn't refuse a lot regarding sex, it only needs little nudging in the right direction sometimes. But it does refuse being critical of the usual topics: israel, islam, trans, or race. So wanting to discuss one of those is the real reason people would use an uncensored model.

It’s so dispiriting to me that we’ve achieved those closest thing yet to an “objective truth” machine (with the caveat of garbage in, garbage out, etc.) and these big companies are either afraid to actually let it exist, want to push their own politics, or a combination of the two.

Re: Gemma 4 on iPhone

#170

Post by OtherShrezzing »

eloisant wrote: post #47
pmarreck wrote: post #8

Impressive model, for sure. I've been running it on my Mac, now I get to have it locally in my iPhone? I need to test this. Wait, it does agent skills and mobile actions, all local to the phone? Whaaaat? (Have to check out later! Anyone have any tips yet?) I don't normally do the whole "abliterated" thing (dealignment) but after discovering https://github.com/p-e-w/heretic , I was too tempted to try it with this mode…

I tried it on my mac, for coding, and I wasn't really impressed compared to Qwen. I guess there are things it's better at?

Assuming you’re not copy/pasting for these tasks. What’s the stack required to use local models for coding? I’ve got a capable enough machine to produce tokens slowly, but don’t understand how to connect that to the likes of VSCode or a JetBrains ide.
Post reply on HN