11–20 of 179 posts
Earlier quoted context omitted.
Imagine a model smarter than most humans that fits on your phone. edit: I seem to be the only one excited by the possibilities of such small yet powerful models. This is an iPhone moment: a computer that fits in your pocket, except this time it's smart.
intelligence != memory
Probably a better link: https://developers.googleblog.com/en/introducing-gemma-3n/ Gemma 3n is a model utilizing Per-Layer Embeddings to achieve an on-device memory footprint of a 2-4B parameter model. At the same time, it performs nearly as well as Claude 3.7 Sonnet in Chatbot Arena.
That seems way too good to be true. What's the catch?
On the other hand, I'm really not looking forward to app sizes ballooning even more – there's no reasonable way to share them across apps at least on iOS, and I can absolutely imagine random corporate apps to start including LLMs, just because it's possible.
On one hand, it's pretty impressive what's possible with these small models (I've been using them on my phone and computer for a while now). On the other hand, I'm really not looking forward to app sizes ballooning even more – there's no reasonable way to share them across apps at least on iOS, and I can absolutely imagine random corporate apps to start including LLMs, just because it's possible.
Though, I won't be surprised if they try to force devs to use their models for "privacy" (and not monopolistic reasons, of course).
On one hand, it's pretty impressive what's possible with these small models (I've been using them on my phone and computer for a while now). On the other hand, I'm really not looking forward to app sizes ballooning even more – there's no reasonable way to share them across apps at least on iOS, and I can absolutely imagine random corporate apps to start including LLMs, just because it's possible.
That sounds like a problem iOS will eventually deal with, as many apps are going to want this technology, and since Apple distributes apps - they aren't interested in the average app being 10x larger when they could solve the problem easily. Though, I won't be surprised if they try to force devs to use their models for "privacy" (and not monopolistic reasons, of course).
Probably a better link: https://developers.googleblog.com/en/introducing-gemma-3n/ Gemma 3n is a model utilizing Per-Layer Embeddings to achieve an on-device memory footprint of a 2-4B parameter model. At the same time, it performs nearly as well as Claude 3.7 Sonnet in Chatbot Arena.
That seems way too good to be true. What's the catch?
I can give more details if you (or anyone else!) is interested.
TL;DR: it is scoring only for "How authoritative did the answer look? How much flattering & emojis?"
What is "Per Layer Embeddings"? The only hit I can find for that term is the announcement blogpost. And for that matter, what is >mix’n’match capability in Gemma 3n to dynamically create submodels It seems like mixture-of-experts taken to the extreme, where you actually create an entire submodel instead of routing per token?
> Gemma 3n leverages a Google DeepMind innovation called Per-Layer Embeddings (PLE) that delivers a significant reduction in RAM usage.
Like you I’m also interested in the architectural details. We can speculate but we’ll probably need to wait for some sort of paper to get the details.
Probably a better link: https://developers.googleblog.com/en/introducing-gemma-3n/ Gemma 3n is a model utilizing Per-Layer Embeddings to achieve an on-device memory footprint of a 2-4B parameter model. At the same time, it performs nearly as well as Claude 3.7 Sonnet in Chatbot Arena.
Imagine a model smarter than most humans that fits on your phone. edit: I seem to be the only one excited by the possibilities of such small yet powerful models. This is an iPhone moment: a computer that fits in your pocket, except this time it's smart.
I'd love to see this deployable to edge that have a Google Coral TPU