Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 270M: Compact model for hyper-efficient AI

developers.googleblog.com

Post reply on HN
Print view View original

51–60 of 325 posts

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#51

Post by NorwegianDude »

canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

The Gemma 3 models are great! One of the few models that can write Norwegian decently, and the instruction following is in my opinion good for most cases. I do however have some issues that might be related to censorship that I hope will be fixed if there is ever a Gemma 4. Maybe you have some insight into why this is happening?

I run a game when players can post messages, it's a game where players can kill each other, and people often send threats along the lines of "I will kill you". Telling Gemma that it should classify a message as game related or a real life threat, and that it is for a message in a game where players can kill each other and threats are a part of the game, and that it should mark it as game related if it is unclear if the message is a game related threat or a real life threat does not work well. For other similar tasks it seems to follow instructions well, but for serious topics it seems to be very biased, and often err on the side of caution, despite being told not to. Sometimes it even spits out some help lines to contact.

I guess this is because it was trained to be safe, and that affects it's ability to follow instructions for this? Or am I completely off here?

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#53

Post by yifanl »

marinhero wrote: post #14
simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

Serious question but if it hallucinates about almost everything, what's the use case for it?

It's funny. Which is subjective, but if it fits for you, it's arguably more useful than Claude.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#54

Post by simonw »

0x00cl wrote: post #46
simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

I see you are using ollamas ggufs. By default it will download Q4_0 quantization. Try `gemma3:270m-it-bf16` instead or you can also use unsloth ggufs `hf.co/unsloth/gemma-3-270m-it-GGUF:16` You'll get better results.

Good call, I'm trying that one just now in LM Studio (by clicking "Use this model -> LM Studio" on https://huggingface.co/unsloth/gemma-3-270m-it-GGUF and selecting the F16 one).

(It did not do noticeably better at my pelican test).

Actually it's worse than that, several of my attempts resulted in infinite loops spitting out the same text. Maybe that GGUF is a bit broken?

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#55

Post by turnsout »

simonw wrote: post #31
marinhero wrote: post #14

Earlier quoted context omitted.

Serious question but if it hallucinates about almost everything, what's the use case for it?

Fine-tuning for specific tasks. I'm hoping to see some good examples of that soon - the blog entry mentions things like structured text extraction, so maybe something like "turn this text about an event into an iCal document" might work?

Google helpfully made some docs on how to fine-tune this model [0]. I'm looking forward to giving it a try!

  [0]: https://ai.google.dev/gemma/docs/core/huggingface_text_full_finetune

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#56

Post by CuriouslyC »

simonw wrote: post #31
marinhero wrote: post #14

Earlier quoted context omitted.

Serious question but if it hallucinates about almost everything, what's the use case for it?

Fine-tuning for specific tasks. I'm hoping to see some good examples of that soon - the blog entry mentions things like structured text extraction, so maybe something like "turn this text about an event into an iCal document" might work?

Fine tuning messes with instruction following and RL'd behavior. I think this is mostly going to be useful for high volume pipelines doing some sort of mundane extraction or transformation.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#58

Post by jtbayly »

simonw wrote: post #36
jtbayly wrote: post #24

Can somebody give me a link to a tutorial on how I would go about fine-tuning this? Also, what sorts of things might I consider fine-tuning it for?

This tutorial looks useful: https://ai.google.dev/gemma/docs/core/huggingface_text_full_...

Just what I was looking for. Thanks!

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#59

Post by jtbayly »

JLCarveth wrote: post #40
jtbayly wrote: post #24

Can somebody give me a link to a tutorial on how I would go about fine-tuning this? Also, what sorts of things might I consider fine-tuning it for?

This was linked at the end of Google's announcement: https://docs.unsloth.ai/basics/gemma-3-how-to-run-and-fine-t... Not sure how much data is needed to realistically fine-tune something like this and get useful output.

That doesn’t really show me how to do fine-tuning, but there is a link to a notebook in there that does. Thanks!

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#60

Post by deepsquirrelnet »

whinvik wrote: post #23

Curious. Are there real world usecases where people have finetuned such tiny models and put them into production.

I’m not sure what I’d use them for, except maybe tag generation? Encoders of this size usually outperform by a wide margin on tasks they would overlap with.
Post reply on HN