Gemma 3n preview: Mobile-first AI • Hacker News Board

Re: Gemma 3n preview: Mobile-first AI

#151

Post by m3kw9 » Wed, May 21, 2025, 2:47 PM UTC

philipkglass wrote: post #76

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

I assume that "pretty fast" depends on the phone. My old Pixel 4a ran Gemma-3n-E2B-it-int4 without problems. Still, it took over 10 minutes to finish answering "What can you see?" when given an image from my recent photos. Final stats: 15.9 seconds to first token 16.4 tokens/second prefill speed 0.33 tokens/second decode speed 662 seconds to complete the answer

10min and 10% battery?

Re: Gemma 3n preview: Mobile-first AI

#152

Post by sebastiennight » Wed, May 21, 2025, 3:10 PM UTC

username135 wrote: post #142

Ive been using the text-to-speech model Whisper, from fdriod. Its rather small and all processing is done locally on my phone. Its pretty good.

You mean speech-to-text, right? For dictation/transcription?

It is pretty good indeed (despite the ~30sec input limit), but this feels unrelated to the topic at hand.

Re: Gemma 3n preview: Mobile-first AI

#153

Post by GTP » Wed, May 21, 2025, 3:43 PM UTC

anonzzzies wrote: post #138

rhdjsjebshjffn wrote: post #32

Earlier quoted context omitted.

I can't speak for anyone else, but these models only seem about as smart as google search, with enormous variability. I can't say I've ever had an interaction with a chatbot that's anything redolent of interaction with intelligence. Now would I take AI as a trivia partner? Absolutely. But that's not really the same as what I look for in "smart" humans.

>anything redolent of interaction with intelligence compared to what you are used to right? I know it's elitist but most people My circle of people I talk with during the day has changed since I took on more charity which consists of fixing up old laptops and installing Ubuntu on them; I get them for free from everyone and I give them to people who cannot afford, including some lessons and remote support (which is ea…

Judging from your comment, it seems that your statistical sample is heavily biased as well, as you are interacting with people that can't afford a laptop. That's not representative of the average person.

Re: Gemma 3n preview: Mobile-first AI

#154

Post by shepherdjerred » Wed, May 21, 2025, 3:51 PM UTC

grav wrote: post #104

android521 wrote: post #102

They should ship a model within the chrome browser. So developers can just call api to access the model for their apps. It seems like a great idea. Don't know why they are not doing it yet.

It seems they are: https://developer.chrome.com/docs/ai/built-in

Really excited to see this shipped & hopefully get cross-browser support

Re: Gemma 3n preview: Mobile-first AI

#155

Post by thomashop » Wed, May 21, 2025, 4:01 PM UTC

rao-v wrote: post #111

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

Why are we still launching models without simple working python example code (or llama.cpp support)?

Who runs python code on mobile?

Re: Gemma 3n preview: Mobile-first AI

#156

Post by sogen » Wed, May 21, 2025, 4:04 PM UTC

bionhoward wrote: post #83

Anybody know a good way to try this model on iPhone?

I think with the Mollama app, just tried it but these latest models are not visible in the list yet.

Re: Gemma 3n preview: Mobile-first AI

#157

Post by nicholasjarnold » Wed, May 21, 2025, 4:53 PM UTC

andrepd wrote: post #125

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

Suggest giving it no networking permissions (if indeed this is about on-device AI).

Networking perms seem to be required on initial startup of the app.

I just installed the apk on a GrapheneOS endpoint (old Pixel 7 Pro) without the Google Play Services installed. The app requires network access to contact Hugging Face and download the model through your HF account. It also requires some interaction/permission agreement with Kaggle. Upon install _with_ network perms the app works, and I'm getting decent performance on the Gemma-3n-E2B-it-int4 model (5-6 token/s). Ok, cool.

Now kill the app, disable network permissions and restart it. Choose one of the models that you downloaded when it had network access. It still works. It does appear to be fully local. Yay.

Re: Gemma 3n preview: Mobile-first AI

#158

Post by [deleted] » Wed, May 21, 2025, 5:06 PM UTC

rvnx wrote: post #130

Absolute shit. Comparing it to Sonnet 3.7 is an insult. # Is Eiffel Tower or a soccer ball bigger ? > A soccer ball is bigger than the Eiffel Tower! Here's a breakdown: > Eiffel Tower: Approximately 330 meters (1,083 feet) tall. > Soccer Ball: A standard soccer ball has a circumference of about 68-70 cm (27-28 inches). > While the Eiffel Tower is very tall, its base is relatively small compared to its height. A socce…

[deleted]

Re: Gemma 3n preview: Mobile-first AI

#159

Post by TiredOfLife » Wed, May 21, 2025, 5:37 PM UTC

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

Is there a list of which SOCs support the GPU acceleration?

Re: Gemma 3n preview: Mobile-first AI

#160

Post by tgtweak » Wed, May 21, 2025, 5:46 PM UTC

ljosifov wrote: post #42

On Hugging face I see 4B and 2B versions now - https://huggingface.co/collections/google/gemma-3n-preview-6... Gemma 3n Preview google/gemma-3n-E4B-it-litert-preview google/gemma-3n-E2B-it-litert-preview Interesting, hope it comes on LMStudio as MLX or GGUF. Sparse and or MoE models make a difference when running on localhost. MoE Qwen3-30B-A3B most recent game changer for me. Activating only 3b weights on the gpu co…

It's a matter of time before we get a limited activation model for mobile - the main constraint is the raw model size, more than the memory usage. A 4B-A1B should be considerably faster on mobile though, for an equivalent size model (~4Gb).