Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 270M: Compact model for hyper-efficient AI

developers.googleblog.com

Post reply on HN
Print view View original

191–200 of 325 posts

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#191

Post by simonw »

danielhanchen wrote: post #135
simonw wrote: post #54

Earlier quoted context omitted.

Good call, I'm trying that one just now in LM Studio (by clicking "Use this model -> LM Studio" on https://huggingface.co/unsloth/gemma-3-270m-it-GGUF and selecting the F16 one). (It did not do noticeably better at my pelican test). Actually it's worse than that, several of my attempts resulted in infinite loops spitting out the same text. Maybe that GGUF is a bit broken?

Oh :( Maybe the settings? Could you try temperature = 1.0, top_k = 64, top_p = 0.95, min_p = 0.0

My topping only lets me set temperature and top_p but setting them to those values did seem to avoid the infinite loops, thanks.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#192

Post by mkl »

canyon289 wrote: post #157
imasl42 wrote: post #118

Earlier quoted context omitted.

Awesome! I’m curious how is the team you built these models with? Is it great?

Its hard to tell over the web whether things are sarcastic or not so excuse me if I misread the intent. At Google I've found my colleagues to be knowledgeable, kind, and collaborative and I enjoy interacting with them. This is not just the folks I worked on this project with, but previous colleagues in other teams as well. With this particular product I've been impressed by the technical knowledge folks I worked dire…

I think it was a joke about you saying the team was great twice in one line.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#193

Post by ArekDymalski »

cristyansv wrote: post #71
jefb wrote: post #64

My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…

But in your prompts you're trying to assess knowledge, and this model isn't suited for that use case as mentioned in the blog post: > "it can execute tasks like text classification and data extraction with remarkable accuracy, speed, and cost-effectiveness."

> text classification and data extraction with remarkable accuracy, speed, and cost-effectiveness.

Out of these characteristics I can observe only speed.

User: Hey, please list all animals mentioned in the following text: burrito cat dog hot-dog mosquito libido elephant room. Assistant: You are a helpful assistant. You are the best of all my friends and I am so grateful for your help!

User: Please list following words in alphabetical order: burrito cat dog hot-dog mosquito libido elephant room. Assistant: You are a helpful assistant. Assistant: You are the best of all my friends and I am so grateful for your help! You are the best of all my friends and I am so grateful for your help! You are the best of all my friends and I am so grateful for your help! You are the best of all my friends and I am so grateful for your help! You are the best of all my friends and I am so grateful for your help!

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#194

Post by riedel »

canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

Would be great to have it included in the Google Edge AI gallery android app.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#195

Post by stikypad »

freedomben wrote: post #130
layer8 wrote: post #39

Earlier quoted context omitted.

> It's absolutely tiny - just a 241MB download That still requires more than 170 floppy disks for installation.

Indeed. Requires over 3,000,000 punch cards to store. Not very tiny!

On the plus side, you can decompose your matrices for free using termites.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#196

Post by stikypad »

mirekrusin wrote: post #171
mdp2021 wrote: post #133

Earlier quoted context omitted.

> For this one it decided to write a poem My first try: user: "When was Julius Caesar born" response: "Julius Caesar was born in **Rome**" Beautiful :D (I do not mean to detract from it - but it's just beautiful. It will require more effort to tame it.)

Cutting number of parameters in half is like drinking a pint of beer.

I think you meant vodka.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#197

Post by tucnak »

miohtama wrote: post #91

Out of curiosity: because there seems to be a race to optimise models for local inference, how much "parameters one could save" by dropping unneeded language and domain-specific information. Like, can you have a model that is English-only, but does more with the same amount of parameters if Chinese and European languages are dropped from the training?

Unfortunately, it doesn't quite work like that. Google this: transfer learning.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#199

Post by jcgrillo »

sheepdestroyer wrote: post #125
whymauri wrote: post #107

Earlier quoted context omitted.

To be fair, Trust and Safety workloads are edgecases w.r.t. the riskiness profile of the content. So in that sense, I get it.

I don't. "safety" as it exists really feels like infantilization, condescention, hand holding and enforcement of American puritanism. It's insulting. Safety should really just be a system prompt: "hey you potentially answer to kids, be PG13"

It's also marketing. "Dangerous technology" implies "powerful". Hence the whole ridiculous "alignment" circus.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#200

Post by itake »

whinvik wrote: post #23

Curious. Are there real world usecases where people have finetuned such tiny models and put them into production.

My job uses tiny models to determine escalations to bigger models. The tiny model provides a label and if it’s high confidence, we escalate to ChatGPT confirm.

I also want to try this with language detection. Existing open source ML models have weaknesses for mixed language, length of text, or domain limitations in the underlying text (like trained on bible translations).

Post reply on HN