Gemma 3 270M: Compact model for hyper-efficient AI • Hacker News Board

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#21

Post by ertgbnm » Thu, Aug 14, 2025, 5:24 PM UTC

simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

He may generate useless tokens but boy can he generate ALOT of tokens.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#22

Post by jasonjmcghee » Thu, Aug 14, 2025, 5:24 PM UTC

I'm _very_ interested to see what this can be fine-tuned to do.

I've heard folks say a number of times that neuromuscular control / locomotion (or w/e) are hundreds of millions of parameters rather than billions.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#23

Post by whinvik » Thu, Aug 14, 2025, 5:26 PM UTC

Curious. Are there real world usecases where people have finetuned such tiny models and put them into production.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#24

Post by jtbayly » Thu, Aug 14, 2025, 5:27 PM UTC

Can somebody give me a link to a tutorial on how I would go about fine-tuning this?

Also, what sorts of things might I consider fine-tuning it for?

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#25

Post by ants_everywhere » Thu, Aug 14, 2025, 5:27 PM UTC

rotexo wrote: post #17

marinhero wrote: post #14

Earlier quoted context omitted.

Serious question but if it hallucinates about almost everything, what's the use case for it?

An army of troll bots to shift the Overton Window?

oh no now we'll never hear the end of how LLMs are just statistical word generators

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#26

Post by 0x457 » Thu, Aug 14, 2025, 5:27 PM UTC

chrismustcode wrote: post #8

ActorNightly wrote: post #3

Earlier quoted context omitted.

How does the 270 perform with coding? I use Gemma27b currently with a custom agent wrapper and its working pretty well.

I’d be stunned if a 270m model could code with any proficiency. If you have an iPhone with the semi-annoying autocomplete that’s a 34m transformer. Can’t imagine a model (even if it’s a good team behind it) to do coding with 8x the parameters of a next 3/4 word autocomplete.

Someone should try this on that model: https://www.oxen.ai/blog/training-a-rust-1-5b-coder-lm-with-...

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#27

Post by iLoveOncall » Thu, Aug 14, 2025, 5:27 PM UTC

marinhero wrote: post #14

simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

Serious question but if it hallucinates about almost everything, what's the use case for it?

Nothing, just like pretty much all models you can run on consumer hardware.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#28

Post by cyanydeez » Thu, Aug 14, 2025, 5:27 PM UTC

simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

the question is wheather you can make a fine tuned version and spam any given forum within an hour with the most attuned but garbage content.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#29

Post by cyanydeez » Thu, Aug 14, 2025, 5:28 PM UTC

iLoveOncall wrote: post #27

marinhero wrote: post #14

Earlier quoted context omitted.

Serious question but if it hallucinates about almost everything, what's the use case for it?

Nothing, just like pretty much all models you can run on consumer hardware.

This message brought to you by OpenAI: we're useless, but atleast theres a pay gate indicating quality!

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#30

Post by striking » Thu, Aug 14, 2025, 5:28 PM UTC

marinhero wrote: post #14

simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

Serious question but if it hallucinates about almost everything, what's the use case for it?

It's intended for finetuning on your actual usecase, as the article shows.