Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 270M: Compact model for hyper-efficient AI

developers.googleblog.com

Post reply on HN
Print view View original

231–240 of 325 posts

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#231

Post by peter492927 »

canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

Thank you a lot for working on these models! If you think it would make sense, I think a bigger sized Gemma model would be really cool. Models in the 70B parameter range can be run at q4 on two 3090 or similar hardware and should offer considerable performance improvement over 27B. There’s also the DGX Spark as a possible target.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#232

Post by K0balt »

littlestymaar wrote: post #108
yomismoaqui wrote: post #102

Earlier quoted context omitted.

Evaluating a 270M model on encyclopedic knowledge is like opening a heavily compressed JPG image and saying "it looks blocky"

What I read above is not an evaluation on “encyclopedic knowledge” though, it's a very basic a common sense: I wouldn't mind if the model didn't know the name of the biggest mountain on earth, but if the model cannot grasp the fact that the same mountain cannot simultaneously be #1, #2 and #3, then the model feels very dumb.

It gave you the tallest mountain every time. You kept asking it for various numbers of “tallest mountains” and each time it complied.

You asked it to enumerate several mountains by height, and it also complied.

It just didn’t understand that when you said the 6 tallest mountains that you didn’t mean the tallest mountain, 6 times.

When you used clearer phrasing it worked fine.

It’s 270m. It’s actually a puppy. Puppies can be trained to do cool tricks, bring your shoes, stuff like that.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#233

Post by ramoz »

AJRF wrote: post #101

I've got a very real world use case I use DistilBERT for - learning how to label wordpress articles. It is one of those things where it's kind of valuable (tagging) but not enough to spend loads on compute for it. The great thing is I have enough data (100k+) to fine-tune and run a meaningful classification report over. The data is very diverse, and while the labels aren't totally evenly distributed, I can deal with…

Please provide updates when you have them.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#234

Post by yborg »

andrehacker wrote: post #144
mrcwinn wrote: post #52

Apple should be doing this. Unless their plan is to replace their search deal with an AI deal -- it's just crazy to me how absent Apple is. Tim Cook said, "it's ours to take" but they really seem to be grasping at the wind right now. Go Google!

As every other thread about LLMs here on HN points out: LLMs are stupid and useless as is. While I don't agree with that sentiment, no company has yet found a way to "do it right" to the extent that investments are justified in the long run. Apple has a history of "being late" and then obliterating the competition with products that are way ahead the early adopters (e.g. MP3 players, smart phones, smart watches).

Yes, Vision Pro has really solved VR.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#236

Post by canyon289 »

nh43215rgb wrote: post #228
rao-v wrote: post #221

Earlier quoted context omitted.

Not at Google (anymore), but Gemma3n is a radically different (and very cool) architecture. The MatFormer approach essentially lets you efficiently change how many parameters of the model you use while inferencing. The 2B model they released is just the sub model embedded in the original 4B model. You can also fiddle with the model and pull a 2.5 or 3B version pu also! This is a more traditional LLM architecture (lik…

oh ok thank you. so something like MoE? That might not be so correct but at least the models need different architecture(MatFormer) to be classified under gemma3n.

Its not an MOE, its what's referred to as a dense architecture, same as the Gemma3 models (But not 3n as noted)

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#237

Post by weird-eye-issue »

AJRF wrote: post #101

I've got a very real world use case I use DistilBERT for - learning how to label wordpress articles. It is one of those things where it's kind of valuable (tagging) but not enough to spend loads on compute for it. The great thing is I have enough data (100k+) to fine-tune and run a meaningful classification report over. The data is very diverse, and while the labels aren't totally evenly distributed, I can deal with…

It's going to perform badly unless you have very few tags and it's easy to classify them

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#238

Post by gf000 »

ComputerGuru wrote: post #121
gnulinux wrote: post #78

Earlier quoted context omitted.

Well, this is a 270M model which is like 1/3 of 1B parameters. In the grand scheme of things, it's basically a few matrix multiplications, barely anything more than that. I don't think it's meant to have a lot of knowledge, grammar, or even coherence. These input: ``` Customer Review says: ai bought your prod-duct and I wanna return becaus it no good. Prompt: Create a JSON object that extracts information about this…

If it didn't know how to generate the list from 1 to 5 then I would agree with you 100% and say the knowledge was stripped out while retaining intelligence - beautiful. But the fact that it does, but cannot articulate the (very basic) knowledge it has *and* in the same chat context when presented with (its own) list of mountains from 1 to 5 that it cannot grasp it made a LOGICAL (not factual) error in repeating the r…

Have you used an LLM? I mean the actual large models? Because they do the exact same errors, just on a slightly less frequent/better hidden manner.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#239

Post by katzenversteher »

sheepdestroyer wrote: post #125
whymauri wrote: post #107

Earlier quoted context omitted.

To be fair, Trust and Safety workloads are edgecases w.r.t. the riskiness profile of the content. So in that sense, I get it.

I don't. "safety" as it exists really feels like infantilization, condescention, hand holding and enforcement of American puritanism. It's insulting. Safety should really just be a system prompt: "hey you potentially answer to kids, be PG13"

I also don't get it. I mean if the training data is publicly available, why isn't that marked as dangerous? If the training data contains enough information to roleplay a killer or a hooker or build a bomb, why is the model censored?

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#240

Post by katzenversteher »

canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

I was wondering the whole time why people in the comments are so hyped about this, then I finally noticed (after I stumbled upon a comment about running this on a mobile phone) that it's "270M" not "270B" model :)
Post reply on HN