Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 270M: Compact model for hyper-efficient AI

developers.googleblog.com

Post reply on HN
Print view View original

171–180 of 325 posts

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#171

Post by mirekrusin »

mdp2021 wrote: post #133
simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

> For this one it decided to write a poem My first try: user: "When was Julius Caesar born" response: "Julius Caesar was born in **Rome**" Beautiful :D (I do not mean to detract from it - but it's just beautiful. It will require more effort to tame it.)

Cutting number of parameters in half is like drinking a pint of beer.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#172

Post by canyon289 »

ankit219 wrote: post #143
canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

This is super cool. Usually you dont see effective models at 270M out in the wild. The architectural choices are new and interesting as well. Would it be okay for you to divulge some more training information here? With 170M embedding parameters, how do you ensure no embedding collapse and keeping the embedding matrix stable at training time? (i know i am asking too much, but just curious). There is a clear trade off…

Not rude at all and I'll again share what I can.

We ran a bunch of experimental architectures at this size to get a sense of performance at this size, in particular how well it was able to adapt to datasets across some loss measures.

For the embedding size it comes from a mix of "hard technical" data, like the loss measures I mentioned above, and for this model it also comes from community considerations such as adaptability across input tokens and consistency with the gemma ecosystem. At this size you are right its a bit funny the embedding is so large.

For more details read the Gemma3 technical report https://arxiv.org/pdf/2503.19786. It doesnt cover the 270m model as this was written from the 1b to 27b gemma3 release but itll answer some of your questions. As for 270m we may share more information in the future, Up until now we were just focused on getting the model out there.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#173

Post by mirekrusin »

marinhero wrote: post #14
simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

Serious question but if it hallucinates about almost everything, what's the use case for it?

The same as having a goldfish. You can train it to do a trick I guess.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#174

Post by dileeparanawake »

canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

This is cool. For on device models any plans / models that use MOE in relatively resource constrained setups (I’m thinking MBP M1 16gb ram)? I’m using LM studio but all the Gemma models (mlx) seem to crash but surprisingly managed to get gpt-oss 20b working (slow) on my mbp.

I find performance in resource constrained environments interesting.

In particular trying to find decent code models (on device backup) but also tts applications and voice to text.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#175

Post by dismalaf »

thegeomaster wrote: post #158
dismalaf wrote: post #63

Earlier quoted context omitted.

I'm making an app where literally all I want to do with an LLM is generate tags. This model has failed with flying colours, literally takes forever to parse anything and doesn't follow instructions. Edit - I should add, currently the model I'm using is Gemini Flash Lite through the Gemini API. It's a really good combo of fast, follows instructions, gives correct results for what I want and cost-effective. I still wou…

Well, Gemini Flash Lite is at least one, or likely two orders of magnitude larger than this model.

That's fair but one can dream of being able to simply run a useful LLM on CPU on your own server to simplify your app and save costs...

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#176

Post by canyon289 »

andrewstuart wrote: post #62
canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

What effort do you folks take to see your models actually running on hardware such as AMD Strix Halo or Apple M3M4? I get the sense that AI is at the “hobby kit computing” stage where they used to dump all the components in a box and give you a schematic and a soldering iron and happily say “you make it work!” And that worked in the early days of computing because there was a small number of people really motivated f…

I don't think we dumped the model and say you do the rest?

Myself and my colleagues spent many days transforming the weights into various open compatible formats. And its not just us there's many orgs and partners dedicating their time, resources, and companies to making all open models easy to use.

I encourage you to explore the solutions provided by them. We linked some in our blog post here, and there's more. They've all done a fantastic job building frankly an insane amount of infrastructure, documentation, and community support in the last 2+ years. Some of them are here in this HN thread answering questions.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#177

Post by schyzomaniac »

canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

hi, congrats for the amazing work!

i love the 27b model, and i use it basically daily. however when i tried to finetune it for a task in a low resource language, unfortunately i did not succeed: lora just did not picked up the gist of the task, full finetune lead to catastrophic forgetting.

may i ask four your advice, or do you have any general tips how to do that properly?

thanks in advance for your help :)

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#178

Post by canyon289 »

metalliqaz wrote: post #117

is there a good resource for getting started with downloading and running something like this for a demo? There are just so many tools/platforms in the mix now it makes my head spin.

The blog post contains links to several ways to try this model, locally, on colab, and in the cloud. Pick what works best for you!

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#179

Post by danielhanchen »

canyon289 wrote: post #160
danielhanchen wrote: post #135

Earlier quoted context omitted.

Oh :( Maybe the settings? Could you try temperature = 1.0, top_k = 64, top_p = 0.95, min_p = 0.0

Daniel, thanks for being here providing technical support as well. Cannot express enough how much we appreciate your all work and partnership.

Thank you and fantastic work with Gemma models!

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#180

Post by canyon289 »

blitzar wrote: post #111
canyon289 wrote: post #85

Earlier quoted context omitted.

Thank you Jeffrey, and we're thrilled that you folks at Ollama partner with us and the open model ecosystem. I personally was so excited to run ollama pull gemma3:270b on my personal laptop just a couple of hours ago to get this model on my devices as well!

> gemma3:270b I think you mean gemma3:270m - Its Dos Comas not Tres Comas

Ah yes thank you. Even I still instinctively type B
Post reply on HN