Gemma 3 Technical Report [pdf]

Post by diggan » Wed, Mar 12, 2025, 4:05 PM UTC

whimsicalism wrote: post #176

diggan wrote: post #173

Earlier quoted context omitted.

Regardless of where you get the weights, Google says you need to follow their terms and conditions for the model/weights: > By using, reproducing, modifying, distributing, performing or displaying any portion or element of Gemma, Model Derivatives including via any Hosted Service, (each as defined below) (collectively, the "Gemma Services") or otherwise accepting the terms of this Agreement, you agree to be bound by…

i think generally these companies are too afraid of the obvious rejoinder to try actually enforcing these terms

Post by traceroute66 » Wed, Mar 12, 2025, 4:30 PM UTC

jampekka wrote: post #101

Per quick testing the 27b model seems very strong at least in natural language. It produces even good Finnish, in which smaller models tend to really struggle. Very promising. Edit: Per even quicker testing the Finnish language performance degrades rapidly with the smaller models, as is usually the case. Would be great to have language specific distillations from larger models.

Post by bguberfain » Wed, Mar 12, 2025, 4:44 PM UTC

alekandreev wrote: post #63

magicalhippo wrote: post #54

Earlier quoted context omitted.

Thanks, been using Gemma 2 a lot at home as it still holds up very well and the 9B version runs great on my 2080Ti. Strong prompt adherence coupled with overall capability makes it very useful. Looking forward to trying Gemma 3. I have some dumb questions though, might as well ask. How do you decide on the model sizes? And how do you train them? Independently or are they related somehow?

Picking model sizes is not an exact science. We look for sizes that will fit quantized on different categories on devices (e.g., low-end and high-end smartphone, laptops and 16GB GPUs, and bigger GPUs/TPUs). We also want the ratio of model width to depth (number of layers) to be consistently around 90, which we found works best. The models are trained with distillation from a bigger teacher. We train them independent…

Post by numpad0 » Wed, Mar 12, 2025, 4:54 PM UTC

miki123211 wrote: post #70

LeoPanthera wrote: post #13

> They are designed to help prevent our models from generating harmful content, i.e., > [...] > Sexually explicit content Dear tech companies. Sexually explicit content is not harmful. Why are you all run by puritans? I don't even want to make edgy porn, I just want to be treated like an adult.

It's harmful in that there exists a significant and vocal subset of users who does not wish to see that content or does not wish their children to do so. It's easier to teach your model never to produce that kind of content than to teach it to perfectly distinguish whether this user should see that content or not. TV channels are barred from broadcasting this kind of content for similar reasons. Sure, there are alway…

Post by genpfault » Wed, Mar 12, 2025, 5:07 PM UTC

meetpateltech wrote: post #2

Gemma 3 is out! Multimodal (image + text), 128K context, supports 140+ languages, and comes in 1B, 4B, 12B, and 27B sizes with open weights & commercial use. Gemma 3 model overview: https://ai.google.dev/gemma/docs/core Huggingface collection: https://huggingface.co/collections/google/gemma-3-release-67... ollama: https://ollama.com/library/gemma3