Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3n preview: Mobile-first AI

developers.googleblog.com

Post reply on HN
Print view View original

81–90 of 179 posts

Re: Gemma 3n preview: Mobile-first AI

#82

Post by [deleted] »

px43 wrote: post #79
KoolKat23 wrote: post #72

Earlier quoted context omitted.

Thanks for this guide it's great. Okay perhaps my phones not great and perhaps this isn't optimized/pruned for phone use but it's unusably slow. The answers are solid from my brief test. I wouldn't exactly say phone use, unless you have no internet and you don't mind a bit of a wait. Really impressive, regardless.

What phone are you using?

[deleted]

Re: Gemma 3n preview: Mobile-first AI

#84

Post by [deleted] »

px43 wrote: post #79
KoolKat23 wrote: post #72

Earlier quoted context omitted.

Thanks for this guide it's great. Okay perhaps my phones not great and perhaps this isn't optimized/pruned for phone use but it's unusably slow. The answers are solid from my brief test. I wouldn't exactly say phone use, unless you have no internet and you don't mind a bit of a wait. Really impressive, regardless.

What phone are you using?

[deleted]

Re: Gemma 3n preview: Mobile-first AI

#85

Post by ignoramous »

barnas2 wrote: post #22

Is anyone able to test it via AiStudio? I pay for Google's AI subscription, but any attempt to use this model results in a message telling me I've hit my rate limit.

Someone on r/LocalLLaMa shared this link: https://aistudio.google.com/prompts/new_chat?model=gemma-3n-...

https://www.reddit.com/r/LocalLLaMA/comments/1kr8s40/comment...

Re: Gemma 3n preview: Mobile-first AI

#86

Post by lxgr »

elpakal wrote: post #64
lxgr wrote: post #15

Earlier quoted context omitted.

Given Apple's track record in dealing with the problem of ballooning app sizes, I'm not holding my breath. The incentives are just not aligned – Apple earns $$$ on each GB of extra storage users have to buy.

I don't know how true your comment is about them earning money on each GB, but if you're interested in app size analysis on iOS I made this for that reason https://dotipa.app . I occasionally post decompositions of public .ipa's on the App Store, and I'm looking forward to seeing how these change over the next year.

It seems straightforward to me: Apps take up storage, and the only way to get more of that is to pay Apple's markups, as iOS devices don't support upgradable storage.

On top of the already hefty markup, they don't even take storage capacity into consideration for trade-ins.

Re: Gemma 3n preview: Mobile-first AI

#88

Post by cwoolfe »

bionhoward wrote: post #83

Anybody know a good way to try this model on iPhone?

To use the model on the web to get an idea of its capabilities: https://aistudio.google.com/app/prompts/new_chat?model=gemma... As a software developer, to integrate it into your app? They mention using Google GenAI SDK or MediaPipe: https://firebase.google.com/docs/ai-logic https://ai.google.dev/edge/mediapipe/framework/getting_start... Via downloading an app on the App Store? Sorry, I think you'll just have to wait! ;-)

Re: Gemma 3n preview: Mobile-first AI

#89

Post by resource_waste »

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

It reminds me of GPT3 quality answers. Kind of impressive.

Although my entire usecase of local models is amoral questions, which it blocks. Excited for the abliterated version.

Re: Gemma 3n preview: Mobile-first AI

#90

Post by the_pwner224 »

philipkglass wrote: post #76
nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

I assume that "pretty fast" depends on the phone. My old Pixel 4a ran Gemma-3n-E2B-it-int4 without problems. Still, it took over 10 minutes to finish answering "What can you see?" when given an image from my recent photos. Final stats: 15.9 seconds to first token 16.4 tokens/second prefill speed 0.33 tokens/second decode speed 662 seconds to complete the answer

I did the same thing on my Pixel Fold. Tried two different images with two different prompts: "What can you see?" and "Describe this image"

First image ('Describe', photo of my desk)

- 15.6 seconds to first token

- 2.6 tokens/second

- Total 180 seconds

Second image ('What can you see?', photo of a bowl of pasta)

- 10.3 seconds to first token

- 3.1 tokens/second

- Total 26 seconds

The Edge Gallery app defaults to CPU as the accelerator. Switched to GPU.

Pasta / what can you see:

- It actually takes a full 1-2 minutes to start printing tokens. But the stats say 4.2 seconds to first token...

- 5.8 tokens/second

- 12 seconds total

Desk / describe:

- The output is: while True: print("[toxicity=0]")

- Bugged? I stopped it after 80 seconds of output. 1st token after 4.1 seconds, then 5.7 tokens/second.

Post reply on HN