Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 270M: Compact model for hyper-efficient AI

developers.googleblog.com

Post reply on HN
Print view View original

221–230 of 325 posts

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#221

Post by rao-v »

nh43215rgb wrote: post #190
canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

270M is nice (and rare) addition. Is there a reason why this is not categorized as gemma3n model? I thought small models go under gemma3n category

Not at Google (anymore), but Gemma3n is a radically different (and very cool) architecture. The MatFormer approach essentially lets you efficiently change how many parameters of the model you use while inferencing. The 2B model they released is just the sub model embedded in the original 4B model. You can also fiddle with the model and pull a 2.5 or 3B version pu also!

This is a more traditional LLM architecture (like the original Gemma 3 4B but smaller) and trained on an insane (for the size) number of tokens.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#222

Post by leopoldj »

michelsedgh wrote: post #167

I wish it was multimodal :(

google/gemma-3-4b-it is one of the smallest multimodal models I know. Works well in a 16GB GPU. Works slowly in a 8GB GPU. It can even be fine tuned [1], which where the real power comes from.

1. https://ai.google.dev/gemma/docs/core/huggingface_vision_fin...

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#224

Post by sleepy_keita »

jefb wrote: post #64

My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…

Simulated a RAG, and it still gets it pretty wrong.

> The tallest mountains on Earth, in descending order: 1. Mount Everest, 8849m 2. K2, 8611m 3. Kangchenjunga, 8586m

> Tell me the second tallest mountain on Earth.

The second tallest mountain on Earth is *Mount Everest*.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#225

Post by not_a_bot_4sho »

ertgbnm wrote: post #21
simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

He may generate useless tokens but boy can he generate ALOT of tokens.

> ALOT

'Alot' is not a word. (I made this mistake a lot, too.)

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#227

Post by wanderingmind »

Maybe I'm using it wrong, but when I try to use the full precision FP16 model, load it into chatter UI and ask a simple question,

"write me a template to make a cold call to a potential lead",

It throws me absolute rubbish. On the other hand, Qwen 0.6B Q8 quantized model nails the answer for the same question.

Qwen 0.6B is smaller than gemma full precision. The execution is a tad slow but not by much. I'm not sure why I need to pick a gemma over qwen.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#228

Post by nh43215rgb »

rao-v wrote: post #221
nh43215rgb wrote: post #190

Earlier quoted context omitted.

270M is nice (and rare) addition. Is there a reason why this is not categorized as gemma3n model? I thought small models go under gemma3n category

Not at Google (anymore), but Gemma3n is a radically different (and very cool) architecture. The MatFormer approach essentially lets you efficiently change how many parameters of the model you use while inferencing. The 2B model they released is just the sub model embedded in the original 4B model. You can also fiddle with the model and pull a 2.5 or 3B version pu also! This is a more traditional LLM architecture (lik…

oh ok thank you. so something like MoE? That might not be so correct but at least the models need different architecture(MatFormer) to be classified under gemma3n.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#229

Post by owebmaster »

beefnugs wrote: post #184
canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

This appears to be a new level of "missing the plot" to me. The push to make "ai for everyone" is now just blindly intertwined with hyper specialized "for ai engineers only" releases. Or am I so far behind that "fine tuning your own model" is something a 12 year old who is married to chatGPT does now?

No, it's something a software engineer will do to create an app. React is not enough anymore.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#230

Post by nh43215rgb »

rshemet wrote: post #214
riedel wrote: post #194

Earlier quoted context omitted.

Would be great to have it included in the Google Edge AI gallery android app.

you can run it in Cactus Chat (download from the Play Store)

what model do you input in Cactus Chat? Seems like it's not one of the preset models and ggml-org/gemma-3-270m-GGUF on hf says Note This is a base (pre-trained) model. Do not use for chat!. Is there an alternative model that you can share so that I can put into cactus chat app?
Post reply on HN