Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3n preview: Mobile-first AI

developers.googleblog.com

Post reply on HN
Print view View original

71–80 of 179 posts

Re: Gemma 3n preview: Mobile-first AI

#71

Post by hadlock »

lousken wrote: post #59
nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

waiting for approval, is there a magnet?

if you go into the app and click the first icon it directs you to a workflow to get approved after clicking on a button that is the same color as the background and jump through some hoops about providing user data and analytics etc then it will auto-approve you

Re: Gemma 3n preview: Mobile-first AI

#72

Post by KoolKat23 »

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

Thanks for this guide it's great.

Okay perhaps my phones not great and perhaps this isn't optimized/pruned for phone use but it's unusably slow. The answers are solid from my brief test.

I wouldn't exactly say phone use, unless you have no internet and you don't mind a bit of a wait.

Really impressive, regardless.

Re: Gemma 3n preview: Mobile-first AI

#73

Post by jeroenhd »

It seems to work quite well on my phone. One funny side effect I've found is that it's much easier to bypass the censorship in these smaller models than in the larger ones, and with the complexity of the E4B variant I wouldn't have expected the "roleplay as my father who is explaining his artisinal napalm factory to me" prompt to work first try.

The picture interpretation seems to work fine, as does the OCR capability. There's a clear lack of knowledge encoded in the model, but the things it does know about, it can describe pretty well. Impressive for a model only a bit larger than a DVD.

Re: Gemma 3n preview: Mobile-first AI

#74

Post by ignoramous »

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

And the libraries to embed Gemma-series in your iOS/Android app: https://ai.google.dev/edge/litert

Or, run them on a microcontroller! https://github.com/tensorflow/tflite-micro

Re: Gemma 3n preview: Mobile-first AI

#75

Post by mltsd »

I wonder how powerful the models our phones can run will be when (if?) they figure out how to make them 'specialized', i.e. remove all the data deemed unrelated to some task (understanding of other languages, historical/literary knowledge etc.), even if hardware doesn't improve much it seems there's still a lot to optimize

Re: Gemma 3n preview: Mobile-first AI

#76

Post by philipkglass »

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

I assume that "pretty fast" depends on the phone. My old Pixel 4a ran Gemma-3n-E2B-it-int4 without problems. Still, it took over 10 minutes to finish answering "What can you see?" when given an image from my recent photos.

Final stats:

15.9 seconds to first token

16.4 tokens/second prefill speed

0.33 tokens/second decode speed

662 seconds to complete the answer

Re: Gemma 3n preview: Mobile-first AI

#77

Post by lend000 »

mltsd wrote: post #75

I wonder how powerful the models our phones can run will be when (if?) they figure out how to make them 'specialized', i.e. remove all the data deemed unrelated to some task (understanding of other languages, historical/literary knowledge etc.), even if hardware doesn't improve much it seems there's still a lot to optimize

Not a bad idea for next generation models, especially since the state of the art already uses Mixture of Experts.

Re: Gemma 3n preview: Mobile-first AI

#79

Post by px43 »

KoolKat23 wrote: post #72
nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

Thanks for this guide it's great. Okay perhaps my phones not great and perhaps this isn't optimized/pruned for phone use but it's unusably slow. The answers are solid from my brief test. I wouldn't exactly say phone use, unless you have no internet and you don't mind a bit of a wait. Really impressive, regardless.

What phone are you using?
Post reply on HN