Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3n preview: Mobile-first AI

developers.googleblog.com

Post reply on HN
Print view View original

171–179 of 179 posts

Re: Gemma 3n preview: Mobile-first AI

#171

Post by nolist_policy »

TiredOfLife wrote: post #159
nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

Is there a list of which SOCs support the GPU acceleration?

It uses tflite in the background which can GPU accelerate with OpenGL ES 3.1 or OpenCL[0]. So it should work on pretty much any SOC.

And you really notice that the model is dumber on GPU, because OpenGL doesn't take accuracy that seriously.

[0] https://blog.tensorflow.org/2020/08/faster-mobile-gpu-infere...

Re: Gemma 3n preview: Mobile-first AI

#172

Post by rhdjsjebshjffn »

anonzzzies wrote: post #138
rhdjsjebshjffn wrote: post #32

Earlier quoted context omitted.

I can't speak for anyone else, but these models only seem about as smart as google search, with enormous variability. I can't say I've ever had an interaction with a chatbot that's anything redolent of interaction with intelligence. Now would I take AI as a trivia partner? Absolutely. But that's not really the same as what I look for in "smart" humans.

>anything redolent of interaction with intelligence compared to what you are used to right? I know it's elitist but most people My circle of people I talk with during the day has changed since I took on more charity which consists of fixing up old laptops and installing Ubuntu on them; I get them for free from everyone and I give them to people who cannot afford, including some lessons and remote support (which is ea…

I agree; but what are we supposed to do? Insight has always been paired with the empathy of not having it but

I refuse to touch the IQ bait

Re: Gemma 3n preview: Mobile-first AI

#174

Post by rhdjsjebshjffn »

anonzzzies wrote: post #162
disgruntledphd2 wrote: post #143

Earlier quoted context omitted.

> but most people This is incorrect, IQ tests are normally scaled such that average intelligence is 100, and such that they are approximately normally distributed so that most people will be somewhere between 85-115 (66% on average).

Yep and those people can never 'win' against current llms, let alone future ones. Outside motorcontrol which I specifically excluded. 85 is special housing where I live... LLMs are far beyond that now.

How are you living such that you're regularly pitting humans against computers

Not only is this unbelievable, it's reprehensible

Re: Gemma 3n preview: Mobile-first AI

#175

Post by rhdjsjebshjffn »

onlyrealcuzzo wrote: post #41
rhdjsjebshjffn wrote: post #33

Earlier quoted context omitted.

ML is a kind of memorization, though.

Anything can be a kind of something since that's subjective...

Yea, anything can be a kind of something else. :clown:

Bruh. Do you need to be paid to interact in good faith or were you raised to be social

Re: Gemma 3n preview: Mobile-first AI

#176

Post by devyy »

philipkglass wrote: post #76
nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

I assume that "pretty fast" depends on the phone. My old Pixel 4a ran Gemma-3n-E2B-it-int4 without problems. Still, it took over 10 minutes to finish answering "What can you see?" when given an image from my recent photos. Final stats: 15.9 seconds to first token 16.4 tokens/second prefill speed 0.33 tokens/second decode speed 662 seconds to complete the answer

In my case, it was pretty fast i would say, using S24 Fe, on Gemma3n E2B int 4, it took around 20 seconds to answer "Describe this image". And the result was pretty amazing.

Stats -

CPU -

first token - 4.52 sec

prefill speed - 57.50 sec tokens/s

decode speed - 10.59 tokens/s

Latency - 20.66 sec

GPU -

first token - 1.92 sec

prefill speed - 135.35 sec tokens/s

decode speed - 11.92 tokens/s

Latency - 9.98 sec

Re: Gemma 3n preview: Mobile-first AI

#177

Post by bartaxyz »

gavmor wrote: post #121
angst wrote: post #92

tried out google/gemma-3n-E4B-it-litert-preview on galaxy s25 ultra loads pretty fast. starts to reply near-instant (text chat mode). doesn't answer questions like "when is your cutoff date" apparently answers "may 15 2024" as today date so probably explains why it answered joe biden as answer to who is US president

I always get a little confused when people fact-check bare foundation models. I don't consider them as fact-bearing, but only fact-preserving when grounded in context. Am I missing something?

I guess it's relevant mostly because the general population might use it for that purpose. Having the model do a search instead would be much better.

Wouldn't matter for a tech demo, but once you deploy this on all Android devices, it matters.

Post reply on HN