Gemma 3n preview: Mobile-first AI
developers.googleblog.com
Gemma 3n preview: Mobile-first AI
1–10 of 179 posts
Re: Gemma 3n preview: Mobile-first AI
#2Probably a better link: https://developers.googleblog.com/en/introducing-gemma-3n/
Gemma 3n is a model utilizing Per-Layer Embeddings to achieve an on-device memory footprint of a 2-4B parameter model.
At the same time, it performs nearly as well as Claude 3.7 Sonnet in Chatbot Arena.
Re: Gemma 3n preview: Mobile-first AI
#3Probably a better link: https://developers.googleblog.com/en/introducing-gemma-3n/ Gemma 3n is a model utilizing Per-Layer Embeddings to achieve an on-device memory footprint of a 2-4B parameter model. At the same time, it performs nearly as well as Claude 3.7 Sonnet in Chatbot Arena.
That seems way too good to be true.
What's the catch?
Re: Gemma 3n preview: Mobile-first AI
#4Probably a better link: https://developers.googleblog.com/en/introducing-gemma-3n/ Gemma 3n is a model utilizing Per-Layer Embeddings to achieve an on-device memory footprint of a 2-4B parameter model. At the same time, it performs nearly as well as Claude 3.7 Sonnet in Chatbot Arena.
It's not a 4B parameter model. The E4B variant is 7B parameters with 4B loaded into memory when using per-layer embedding cached to fast storage, and without vision or audio support.
Re: Gemma 3n preview: Mobile-first AI
#5Probably a better link: https://developers.googleblog.com/en/introducing-gemma-3n/ Gemma 3n is a model utilizing Per-Layer Embeddings to achieve an on-device memory footprint of a 2-4B parameter model. At the same time, it performs nearly as well as Claude 3.7 Sonnet in Chatbot Arena.
Imagine a model smarter than most humans that fits on your phone.
edit: I seem to be the only one excited by the possibilities of such small yet powerful models. This is an iPhone moment: a computer that fits in your pocket, except this time it's smart.
Re: Gemma 3n preview: Mobile-first AI
#6What is "Per Layer Embeddings"? The only hit I can find for that term is the announcement blogpost.
And for that matter, what is
>mix’n’match capability in Gemma 3n to dynamically create submodels
It seems like mixture-of-experts taken to the extreme, where you actually create an entire submodel instead of routing per token?
Re: Gemma 3n preview: Mobile-first AI
#7What is "Per Layer Embeddings"? The only hit I can find for that term is the announcement blogpost. And for that matter, what is >mix’n’match capability in Gemma 3n to dynamically create submodels It seems like mixture-of-experts taken to the extreme, where you actually create an entire submodel instead of routing per token?
https://ai.google.dev/gemma/docs/gemma-3n#parameters
> Gemma 3n models are listed with parameter counts, such as E2B and E4B, that are lower than the total number of parameters contained in the models. The E prefix indicates these models can operate with a reduced set of Effective parameters. This reduced parameter operation can be achieved using the flexible parameter technology built into Gemma 3n models to help them run efficiently on lower resource devices.
> The parameters in Gemma 3n models are divided into 4 main groups: text, visual, audio, and per-layer embedding (PLE) parameters. With standard execution of the E2B model, over 5 billion parameters are loaded when executing the model. However, using parameter skipping and PLE caching techniques, this model can be operated with an effective memory load of just under 2 billion (1.91B) parameters, as illustrated in Figure 1.
Re: Gemma 3n preview: Mobile-first AI
#8What is "Per Layer Embeddings"? The only hit I can find for that term is the announcement blogpost. And for that matter, what is >mix’n’match capability in Gemma 3n to dynamically create submodels It seems like mixture-of-experts taken to the extreme, where you actually create an entire submodel instead of routing per token?
https://ai.google.dev/gemma/docs/gemma-3n#parameters > Gemma 3n models are listed with parameter counts, such as E2B and E4B, that are lower than the total number of parameters contained in the models. The E prefix indicates these models can operate with a reduced set of Effective parameters. This reduced parameter operation can be achieved using the flexible parameter technology built into Gemma 3n models to help th…
Thank you, that helped a bit, although it's still not clear what exactly those parameters _are_. "Per-Layer Embedding (PLE) parameters that are used during model execution to create data that enhances the performance of each model layer." is too vague, and I can't find any other reference to "per-layer embedding parameters" in literature.
Re: Gemma 3n preview: Mobile-first AI
#9Probably a better link: https://developers.googleblog.com/en/introducing-gemma-3n/ Gemma 3n is a model utilizing Per-Layer Embeddings to achieve an on-device memory footprint of a 2-4B parameter model. At the same time, it performs nearly as well as Claude 3.7 Sonnet in Chatbot Arena.
Imagine a model smarter than most humans that fits on your phone. edit: I seem to be the only one excited by the possibilities of such small yet powerful models. This is an iPhone moment: a computer that fits in your pocket, except this time it's smart.
intelligence != memory
Re: Gemma 3n preview: Mobile-first AI
#10Earlier quoted context omitted.
https://ai.google.dev/gemma/docs/gemma-3n#parameters > Gemma 3n models are listed with parameter counts, such as E2B and E4B, that are lower than the total number of parameters contained in the models. The E prefix indicates these models can operate with a reduced set of Effective parameters. This reduced parameter operation can be achieved using the flexible parameter technology built into Gemma 3n models to help th…
Thank you, that helped a bit, although it's still not clear what exactly those parameters _are_. "Per-Layer Embedding (PLE) parameters that are used during model execution to create data that enhances the performance of each model layer." is too vague, and I can't find any other reference to "per-layer embedding parameters" in literature.
A layer is a transformer block / layer (basically the building block of the modern LLM architectures) - maybe Gemini can help you: