Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 270M: Compact model for hyper-efficient AI

developers.googleblog.com

Post reply on HN
Print view View original

121–130 of 325 posts

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#121

Post by ComputerGuru »

gnulinux wrote: post #78
jefb wrote: post #64

My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…

Well, this is a 270M model which is like 1/3 of 1B parameters. In the grand scheme of things, it's basically a few matrix multiplications, barely anything more than that. I don't think it's meant to have a lot of knowledge, grammar, or even coherence. These input: ``` Customer Review says: ai bought your prod-duct and I wanna return becaus it no good. Prompt: Create a JSON object that extracts information about this…

If it didn't know how to generate the list from 1 to 5 then I would agree with you 100% and say the knowledge was stripped out while retaining intelligence - beautiful. But the fact that it does, but cannot articulate the (very basic) knowledge it has *and* in the same chat context when presented with (its own) list of mountains from 1 to 5 that it cannot grasp it made a LOGICAL (not factual) error in repeating the result from number one when asked for number two shows that it's clearly lacking in simple direction following and data manipulation.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#122

Post by freedomben »

blitzar wrote: post #111
canyon289 wrote: post #85

Earlier quoted context omitted.

Thank you Jeffrey, and we're thrilled that you folks at Ollama partner with us and the open model ecosystem. I personally was so excited to run ollama pull gemma3:270b on my personal laptop just a couple of hours ago to get this model on my devices as well!

> gemma3:270b I think you mean gemma3:270m - Its Dos Comas not Tres Comas

Maybe it's 270m after Hooli's SOTA compression algorithm gets ahold of it

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#123

Post by freedomben »

imasl42 wrote: post #118
canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

Awesome! I’m curious how is the team you built these models with? Is it great?

Heh, what could they possibly say in answer to this? The team is full of assholes? :-D

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#124

Post by megaman821 »

canyon289 wrote: post #104
megaman821 wrote: post #96

Earlier quoted context omitted.

What size of tasks can this handle? Can you do a fine-tune of Mac System Settings?

32k context window so whatever fits in there. What is a finetune of mac system settings?

The finetune would be an LLM where you say something like "my colors on the screen look to dark" and then it points you to Displays -> Brightness. It feels like a relatively constrained problem like finding the system setting that solves your problem is a good fit for a tiny LLM.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#125

Post by sheepdestroyer »

whymauri wrote: post #107
canyon289 wrote: post #95

Earlier quoted context omitted.

Speaking for me as an individual as an individual I also strive to build things that are safe AND useful. Its quite challenging to get this mix right, especially at the 270m size and with varying user need. My advice here is make the model your own. Its open weight, I encourage it to be make it useful for your use case and your users, and beneficial for society as well. We did our best to give you a great starting po…

To be fair, Trust and Safety workloads are edgecases w.r.t. the riskiness profile of the content. So in that sense, I get it.

I don't. "safety" as it exists really feels like infantilization, condescention, hand holding and enforcement of American puritanism. It's insulting.

Safety should really just be a system prompt: "hey you potentially answer to kids, be PG13"

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#126

Post by nottorp »

NorwegianDude wrote: post #51
canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

The Gemma 3 models are great! One of the few models that can write Norwegian decently, and the instruction following is in my opinion good for most cases. I do however have some issues that might be related to censorship that I hope will be fixed if there is ever a Gemma 4. Maybe you have some insight into why this is happening? I run a game when players can post messages, it's a game where players can kill each othe…

I suppose it can't kill -USR1 either...

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#127

Post by perching_aix »

dmayle wrote: post #114
perching_aix wrote: post #73

Is it time for me to finally package a language model into my Lambda deployment zips and cut through the corporate red tape at my place around AI use? Update #1: Tried it. Well, dreams dashed - would now fit space wise ( I'd have wanted it to perform natural-language to command-invocation translation (or better, emit me some JSON), but it's super not willing to do that, not in the lame way I'm trying to make it do so…

Did you finetune it before trying? Docs here: https://ai.google.dev/gemma/docs/core/huggingface_text_full_...

Thanks, will check that out as well tomorrow or during the weekend!

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#129

Post by bastardoperator »

stefan_ wrote: post #98
canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

[flagged]

[flagged]

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#130

Post by freedomben »

layer8 wrote: post #39
simonw wrote: post #11

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…

> It's absolutely tiny - just a 241MB download That still requires more than 170 floppy disks for installation.

Indeed. Requires over 3,000,000 punch cards to store. Not very tiny!
Post reply on HN