Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3n preview: Mobile-first AI

developers.googleblog.com

Post reply on HN
Print view View original

51–60 of 179 posts

Re: Gemma 3n preview: Mobile-first AI

#51

Post by kcorbitt »

krackers wrote: post #8
onlyrealcuzzo wrote: post #7

Earlier quoted context omitted.

https://ai.google.dev/gemma/docs/gemma-3n#parameters > Gemma 3n models are listed with parameter counts, such as E2B and E4B, that are lower than the total number of parameters contained in the models. The E prefix indicates these models can operate with a reduced set of Effective parameters. This reduced parameter operation can be achieved using the flexible parameter technology built into Gemma 3n models to help th…

Thank you, that helped a bit, although it's still not clear what exactly those parameters _are_. "Per-Layer Embedding (PLE) parameters that are used during model execution to create data that enhances the performance of each model layer." is too vague, and I can't find any other reference to "per-layer embedding parameters" in literature.

I wonder if they've trained the model to operate with a shallower stack; eg. the full model may be composed of 24 transformer blocks, but they've also trained it to accept embeddings at layer 8, so it can be operated with just 16 transformer blocks on lower-resourced devices.

Experimenters in the open source tinkering community have done the opposite (copy/pasting layers in existing models to make them deeper) and it seems to work... fine, with minimal post-training on the new, deeper model required to exceed the performance of the original model. So it's not a crazy idea.

Re: Gemma 3n preview: Mobile-first AI

#52

Post by nolist_policy »

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

Okay from some first tries with story writing, gemma-3n-E4B-it seems to perform between plain Gemma 3 4B and 12B. It definitely retains the strong instruction following which is good.

Hint: You have to set the Max tokens to 32000 for longer conversations. The slider makes it look like it's limited to 1024, just enter it manually.

Re: Gemma 3n preview: Mobile-first AI

#53

Post by viraptor »

quaintdev wrote: post #50

> Gemma 3n enables you to start building on this foundation that will come to major platforms such as Android and Chrome. Seems like we will not be able to run this with Llama and friends. https://developers.googleblog.com/en/introducing-gemma-3n/

What makes you say that? The files can be downloaded, so it will be done. (Maybe the licence will be an issue)

Re: Gemma 3n preview: Mobile-first AI

#54

Post by nolist_policy »

IceWreck wrote: post #24

According to the readme here - https://huggingface.co/google/gemma-3n-E4B-it-litert-preview E4B has a score of 44.4 in the Aider polyglot dashboard. Which means its on-par with gemini-2.5-flash (not the latest preview but the version used for the bench on aider's website), gpt4o and gpt4.5. Thats sounds very good - imagine what a coding focused version of this could do if this is a "generic" embedded only model. On t…

Hmm, the Aider polyglot benchmark has been removed from the huggingface readme.

Also:

> These models were evaluated at full precision (float32)

For 4B effective parameters that's 16 GB ram.

Re: Gemma 3n preview: Mobile-first AI

#55

Post by ankit219 »

yorwba wrote: post #49
ankit219 wrote: post #44

Earlier quoted context omitted.

You can read this for a comprehensive deep dive. https://arxiv.org/pdf/2502.01637 At a very high level, instead of having embeddings at the input layers, this method keeps the embeddings at the layer level. That is every transformer layer would have its own set of learnable embedding vectors that are used to modify the processed hidden states flowing through the network. Mostly, the embeddings are precomputed and sto…

The paper you link to is about a different way to create embeddings at the input layer. In no way does it match your claimed description.

I simplified what i wrote. There is an off accelerator memory where the embeddings are stored and queried at inference time, i did not want to get into details. That is how you reduce the in memory RAM. There are definitely more things going on in the paper as it builds upon the concept I described. The central idea remains the same: you have input embedding layers which map text to continuous vectors. Instead of loading all these layers at runtime, you can break it per layer at training time, and then fetch the required ones from a separate store during inference. Would not be in RAM. Per layer is not mentioned in the paper. But surely it's not a great leap from the paper itself?

Re: Gemma 3n preview: Mobile-first AI

#56

Post by impure »

Interesting that they reduced the memory usage by half. This would address what is IMO the biggest problem with local LLMs: the limited number of parameters resulting in answers that are not very good.

Also it's funny that they are saying that Llama 4 Maverick performs about the same as GPT-4.1 Nano.

Re: Gemma 3n preview: Mobile-first AI

#57

Post by bilbo0s »

lxgr wrote: post #15
onlyrealcuzzo wrote: post #14

Earlier quoted context omitted.

That sounds like a problem iOS will eventually deal with, as many apps are going to want this technology, and since Apple distributes apps - they aren't interested in the average app being 10x larger when they could solve the problem easily. Though, I won't be surprised if they try to force devs to use their models for "privacy" (and not monopolistic reasons, of course).

Given Apple's track record in dealing with the problem of ballooning app sizes, I'm not holding my breath. The incentives are just not aligned – Apple earns $$$ on each GB of extra storage users have to buy.

I was thinking that the entire time I read HN User onlyrealcuzzo's comment.

Why, on Earth, would Apple ever want to solve the problem of Apps taking up more space? That's just not good business. Way better business right now to put R&D into increased memory access speeds.

Apple would need to have a different business model entirely for them to have a business case for fixing this. They may fix it because they just want to help out they AI guys? Maybe in the future they're getting money from the AI guys or something? So fixing it starts to make a lot of sense.

But all other things being equal, the money for Apple is in this not being fixed.

Re: Gemma 3n preview: Mobile-first AI

#58

Post by andy12_ »

yorwba wrote: post #45
andy12_ wrote: post #38

Earlier quoted context omitted.

I think that it's a poorly named reference to this paper [1] that they mention in the blogpost. If I had to give it another more descriptive name, I would probably name it "Per-Layer Embedding Dimensionality" [1] https://arxiv.org/pdf/2310.07707

The MatFormer is clearly called out as a different aspect of the model design. PLE is much more likely to be a reference to the Per-Layer Embeddings paper that will be published in the future once it doesn't give away any secret sauce anymore.

I thought the same, but Per-Layer Embeddings as a name doesn't make sense in any context, and MatFormer does exactly what the blogpost says PLE does. I just think it's more probable that the blogpost was written by several authors and that noone bothered to check the final result.

Re: Gemma 3n preview: Mobile-first AI

#59

Post by lousken »

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

waiting for approval, is there a magnet?

Re: Gemma 3n preview: Mobile-first AI

#60

Post by hmapple »

rhdjsjebshjffn wrote: post #32
esafak wrote: post #5

Earlier quoted context omitted.

Imagine a model smarter than most humans that fits on your phone. edit: I seem to be the only one excited by the possibilities of such small yet powerful models. This is an iPhone moment: a computer that fits in your pocket, except this time it's smart.

I can't speak for anyone else, but these models only seem about as smart as google search, with enormous variability. I can't say I've ever had an interaction with a chatbot that's anything redolent of interaction with intelligence. Now would I take AI as a trivia partner? Absolutely. But that's not really the same as what I look for in "smart" humans.

Have you tried any SOTA models like o3?

If not, I strongly encourage you to discuss your area of expertise with it and rate based on that

It is incredibly competent

Post reply on HN