Gemma 3 Technical Report [pdf] • Hacker News Board

Re: Gemma 3 Technical Report [pdf]

#201

Post by wizee » Wed, Mar 12, 2025, 6:23 PM UTC

tomthe wrote: post #22

Very cool open release. Impressive that a 27b model can be as good as the much bigger state of the art models (according to their table of Chatbot Arena, tied with O1-preview and above Sonnet 3.7). But the example image shows that this model still makes dumb errors or has a poor common sense although it read every information correctly.

It seems to have been very benchmark-tuned for LMArena. In my own experiments, it was roughly in line with other comparably sized models for factual knowledge (like Mistral Small 3), and worse than Mistral Small 3 and Phi-4 at STEM problems and logic. It's much worse than Llama 3.3 70b or Mistral Large 2411 in knowledge or intelligence in reality, even though LMArena ranks it as better than those.

Re: Gemma 3 Technical Report [pdf]

#202

Post by [deleted] » Wed, Mar 12, 2025, 6:29 PM UTC

keheliya wrote: post #198

staticman2 wrote: post #186

Earlier quoted context omitted.

I don't disagree but even Linux has "Terms and conditions" of usage under it's license you really need to dig into what those are. There's no doubt Gemma's license is less permissive than other models and that it has less community finetuners for that reason.

According to the OSI's open source definition, you can't put restrictions against persons or groups or fields of use. In the license, Linux is not restricted in what domain it will be used (good or bad). Here's OSI's argument about this when Meta's llama put such limitations in their license: https://opensource.org/blog/metas-llama-2-license-is-not-ope...

[deleted]

Re: Gemma 3 Technical Report [pdf]

#203

Post by setgree » Wed, Mar 12, 2025, 6:30 PM UTC

meetpateltech wrote: post #2

Gemma 3 is out! Multimodal (image + text), 128K context, supports 140+ languages, and comes in 1B, 4B, 12B, and 27B sizes with open weights & commercial use. Gemma 3 model overview: https://ai.google.dev/gemma/docs/core Huggingface collection: https://huggingface.co/collections/google/gemma-3-release-67... ollama: https://ollama.com/library/gemma3

A kind of ancillary note, but it's amazing to me how fragmented this presentation and documentation is:

* the parent link is to storage.googleapis.com

* There's documentation on ai.google.dev

* The announcement blogpost is https://blog.google/technology/developers/gemma-3/

* you try it on https://aistudio.google.com/

It's helpful to have a top-level post like this, but can some PM please consolidate this into, IDK, ai.google.com/gemini?

Re: Gemma 3 Technical Report [pdf]

#204

Post by yohbho » Wed, Mar 12, 2025, 6:35 PM UTC

danielhanchen wrote: post #123

If it helps anyone, I wrote a detailed analysis here: https://x.com/danielhanchen/status/1899735308180267176 TLDR: 1. 1B text only, 4, 12, 27B Vision + text. 14T tokens 2. 128K context length further trained from 32K. 1B is 32K. 3. Removed attn softcapping. Replaced with QK norm 4. 5 sliding + 1 global attn 5. 1024 sliding window attention 6. RL - BOND, WARM, WARP

[dead]

Re: Gemma 3 Technical Report [pdf]

#205

Post by BoredPositron » Wed, Mar 12, 2025, 6:35 PM UTC

philipkglass wrote: post #197

BoredPositron wrote: post #60

Earlier quoted context omitted.

Everyone is treating this like corps have anything to gain from an open uncensored model. Switch your view and give me a single argument for it? That random nerds on HN stop jerking each other about what „open“ means? You are just not their target group. Having this discussion every time no matter if the model released is censored or not is just insanity. Bring new arguments or don’t use the models you don’t like. Th…

The lack of NSFW knowledge/capability makes them less useful for content moderation. I've tried to use multimodal models for categorizing images from large, mixed data sets. 95% of the input is safe for work. 4% contains nudity but is not sexually explicit. 1% contains nudity and is also sexually explicit. I'd like to categorize content so that nudity is hidden from users by default and that sexually explicit content…

There are models especially tuned for it even open weight ones. llms even multimodal ones are not up to the task. You know what doesn't help the discussion at all? That everyone's response is as usual just about titties.

Re: Gemma 3 Technical Report [pdf]

#206

Post by philipkglass » Wed, Mar 12, 2025, 6:46 PM UTC

BoredPositron wrote: post #205

philipkglass wrote: post #197

Earlier quoted context omitted.

The lack of NSFW knowledge/capability makes them less useful for content moderation. I've tried to use multimodal models for categorizing images from large, mixed data sets. 95% of the input is safe for work. 4% contains nudity but is not sexually explicit. 1% contains nudity and is also sexually explicit. I'd like to categorize content so that nudity is hidden from users by default and that sexually explicit content…

There are models especially tuned for it even open weight ones. llms even multimodal ones are not up to the task. You know what doesn't help the discussion at all? That everyone's response is as usual just about titties.

4 months ago I tried every dedicated NSFW-image-classifier model I could find on HuggingFace or GitHub. They have a high false positive rate on certain kinds of benign content, like close up photographs of hands with painted fingernails, and a high false negative rate on artistic nude photographs. I even tried combining multiple models with gradient boosting but the accuracy barely improved; maybe everyone is training with very similar data sets. At this point I should train my own model but I was hoping to find something capable off-the-shelf, since content moderation is such a common task.

Re: Gemma 3 Technical Report [pdf]

#207

Post by summerlight » Wed, Mar 12, 2025, 6:49 PM UTC

gundmc wrote: post #37

What do companies like Meta and Google gain from releasing open models? Is it just reputational? Attractive to top AI talent?

There are certain demands and if you don't do anything, those will be taken over by competitors and you lose controls. This is especially important for Google as they see LLM as a significant portion of future Cloud business and probably want to have a smooth, exclusive transition path to their proprietary models.

Re: Gemma 3 Technical Report [pdf]

#208

Post by klysm » Wed, Mar 12, 2025, 7:20 PM UTC

setgree wrote: post #203

meetpateltech wrote: post #2

Gemma 3 is out! Multimodal (image + text), 128K context, supports 140+ languages, and comes in 1B, 4B, 12B, and 27B sizes with open weights & commercial use. Gemma 3 model overview: https://ai.google.dev/gemma/docs/core Huggingface collection: https://huggingface.co/collections/google/gemma-3-release-67... ollama: https://ollama.com/library/gemma3

A kind of ancillary note, but it's amazing to me how fragmented this presentation and documentation is: * the parent link is to storage.googleapis.com * There's documentation on ai.google.dev * The announcement blogpost is https://blog.google/technology/developers/gemma-3/ * you try it on https://aistudio.google.com/ It's helpful to have a top-level post like this, but can some PM please consolidate this into, IDK, a…

I don't see how this actually matters - who cares if it it's different top level domains?

Re: Gemma 3 Technical Report [pdf]

#209

Post by ZeroCool2u » Wed, Mar 12, 2025, 7:31 PM UTC

tadamcz wrote: post #191

The launch post for Gemma 3 says: > use Gemma 3 with the Google GenAI SDK https://blog.google/technology/developers/gemma-3/ Does this mean (serverless) API access? I haven't been able to do so or find docs that explain how to.

You can just go here: https://aistudio.google.com/prompts/new_chat

Select Gemma 3 from the drop down on the right side.

Re: Gemma 3 Technical Report [pdf]

#210

Post by homarp » Wed, Mar 12, 2025, 7:38 PM UTC

staticman2 wrote: post #186

diggan wrote: post #173

Earlier quoted context omitted.

Regardless of where you get the weights, Google says you need to follow their terms and conditions for the model/weights: > By using, reproducing, modifying, distributing, performing or displaying any portion or element of Gemma, Model Derivatives including via any Hosted Service, (each as defined below) (collectively, the "Gemma Services") or otherwise accepting the terms of this Agreement, you agree to be bound by…

I don't disagree but even Linux has "Terms and conditions" of usage under it's license you really need to dig into what those are. There's no doubt Gemma's license is less permissive than other models and that it has less community finetuners for that reason.

can you link to Linux terms and conditions? search returned nothing.