Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 270M: Compact model for hyper-efficient AI

developers.googleblog.com

Post reply on HN
Print view View original

131–140 of 325 posts

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#131

Post by hnuser123456 »

jefb wrote: post #64

My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…

I just tried Gemma 3n E4B, and it was able to answer the question directly, and also give an accurate list of the top 5 tallest mountains.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#132

Post by layer8 »

lucb1e wrote: post #38
ertgbnm wrote: post #21

Earlier quoted context omitted.

He may generate useless tokens but boy can he generate ALOT of tokens.

He? I know some Gemmas and it's distinctly a female name; is Gemma a boy's name where you're from?

Doesn’t the “M” in “Gemma 3 270M” Stand for “male”?

Also: https://en.wikipedia.org/wiki/Gemma_Frisius

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#133

Post by mdp2021 »

simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

> For this one it decided to write a poem

My first try:

user: "When was Julius Caesar born"

response: "Julius Caesar was born in **Rome**"

Beautiful :D

(I do not mean to detract from it - but it's just beautiful. It will require more effort to tame it.)

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#134

Post by skybrian »

jefb wrote: post #64

My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…

That’s pretty amusing, but continuing after an error is not worth it. You’re just polluting the context. It’s not going to learn.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#135

Post by danielhanchen »

simonw wrote: post #54
0x00cl wrote: post #46

Earlier quoted context omitted.

I see you are using ollamas ggufs. By default it will download Q4_0 quantization. Try `gemma3:270m-it-bf16` instead or you can also use unsloth ggufs `hf.co/unsloth/gemma-3-270m-it-GGUF:16` You'll get better results.

Good call, I'm trying that one just now in LM Studio (by clicking "Use this model -> LM Studio" on https://huggingface.co/unsloth/gemma-3-270m-it-GGUF and selecting the F16 one). (It did not do noticeably better at my pelican test). Actually it's worse than that, several of my attempts resulted in infinite loops spitting out the same text. Maybe that GGUF is a bit broken?

Oh :( Maybe the settings? Could you try

temperature = 1.0, top_k = 64, top_p = 0.95, min_p = 0.0

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#136

Post by reneberlin »

I am sure with finetuning this can be changed somehow:

(base) ~ ollama run hf.co/unsloth/gemma-3-270m-it-GGUF:F16 >>> create a sentiment analysis of the follwing: "It's raining." The sentiment of the provided text is *negative*.

>>> create a sentiment analysis of the follwing: "It's raining money." The sentiment of the provided text is *negative*.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#137

Post by Balinares »

simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

This is like a kobold to the other models' dragons and I don't hate it. :)

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#138

Post by sogen »

CharlesW wrote: post #18
elAhmo wrote: post #10

Earlier quoted context omitted.

How do you actually run this on an iPhone?

With something like PocketPal AI ( https://github.com/a-ghorbani/pocketpal-ai ). I'd love hear HN'ers opinions on the "best" LM Studio-like option for iOS devices.

For iOS, OpenCat. Has iCloud sync, and one universal app for MacOS and iOS devices.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#139

Post by ungreased0675 »

sheepdestroyer wrote: post #125
whymauri wrote: post #107

Earlier quoted context omitted.

To be fair, Trust and Safety workloads are edgecases w.r.t. the riskiness profile of the content. So in that sense, I get it.

I don't. "safety" as it exists really feels like infantilization, condescention, hand holding and enforcement of American puritanism. It's insulting. Safety should really just be a system prompt: "hey you potentially answer to kids, be PG13"

Safety in the context of LLMs means “avoiding bad media coverage or reputation damage for the parent company”

It has only a tangential relationship with end user safety.

If some of these companies are successful the way they imagine, most of their end users will be unemployed. When they talk about safety, it’s the companies safety they’re referring to.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#140

Post by danielhanchen »

jtbayly wrote: post #59
JLCarveth wrote: post #40

Earlier quoted context omitted.

This was linked at the end of Google's announcement: https://docs.unsloth.ai/basics/gemma-3-how-to-run-and-fine-t... Not sure how much data is needed to realistically fine-tune something like this and get useful output.

That doesn’t really show me how to do fine-tuning, but there is a link to a notebook in there that does. Thanks!

If you need any help on it, ask away!
Post reply on HN