Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 Technical Report [pdf]

storage.googleapis.com

Post reply on HN
Print view View original

111–120 of 260 posts

Re: Gemma 3 Technical Report [pdf]

#111

Post by heinrichf »

alekandreev wrote: post #33

Greetings from the Gemma team! We just got Gemma 3 out of the oven and are super excited to show it to you! Please drop any questions here and we'll answer ASAP. (Opinions our own and not of Google DeepMind.) PS we are hiring: https://boards.greenhouse.io/deepmind/jobs/6590957

I'm comparing Gemma3 12 B (https://ollama.com/library/gemma3; running fully on my 3060 12GB) and Mistral Small 3 24B (https://ollama.com/library/mistral-small; 10% offloaded to the CPU).

- Gemma3 12B: ~100 t/s on prompt eval; 15 t/s on eval

- MistralSmall3 24B: ~500 t/s on prompt eval; 10 t/s on eval

Do you know what different in architecture could make the prompt eval (prefill) so much slower on the 2x smaller Gemma3 model?

Re: Gemma 3 Technical Report [pdf]

#112

Post by YetAnotherNick »

LeoPanthera wrote: post #104
YetAnotherNick wrote: post #28

They say all the models were distilled from a teacher model but they didn't specify what that teacher model is. Interesting thing to hide.

It's a safe bet that it's either one of the Gemini models or a relative of it.

That's what I thought. And it could be pulicity of Gemini as well that it is so good that it can teach students say 5x faster. If it is Gemini, there isn't any reason to hide. My bet is it is some unreleased Gemma or some model.

Re: Gemma 3 Technical Report [pdf]

#113

Post by fragmede »

philipjoubert wrote: post #99
logicchains wrote: post #94

Earlier quoted context omitted.

>It's harmful in that there exists a significant and vocal subset of users who does not wish to see that content or does not wish their children to do so It's hard to think of a scenario where there's a child technical enough to run Gemma 3 locally but somehow unable to access any other written erotica. Project Gutenberg is full of erotic textual content and I haven't heard of anyone calling for that to be banned. >T…

> It's hard to think of a scenario where there's a child technical enough to run Gemma 3 locally but somehow unable to access any other written erotica. The reason you're struggling to understand is that you're thinking about this logically. Adult content is obviously freely available to any child or adult with minimum technical skills. What makes LLMs different is that it's "the new thing" and people respond differe…

Won't somebody think of children‽

Re: Gemma 3 Technical Report [pdf]

#114

Post by andai »

miki123211 wrote: post #70
LeoPanthera wrote: post #13

> They are designed to help prevent our models from generating harmful content, i.e., > [...] > Sexually explicit content Dear tech companies. Sexually explicit content is not harmful. Why are you all run by puritans? I don't even want to make edgy porn, I just want to be treated like an adult.

It's harmful in that there exists a significant and vocal subset of users who does not wish to see that content or does not wish their children to do so. It's easier to teach your model never to produce that kind of content than to teach it to perfectly distinguish whether this user should see that content or not. TV channels are barred from broadcasting this kind of content for similar reasons. Sure, there are alway…

I heard of this described as the minority effect, that a small minority can have a disproportionate impact. The example given is that it's cheaper to make all instances of a product kosher or halal than to make an entirely separate product.

Re: Gemma 3 Technical Report [pdf]

#115

Post by andai »

Al-Khwarizmi wrote: post #109
miki123211 wrote: post #70

Earlier quoted context omitted.

It's harmful in that there exists a significant and vocal subset of users who does not wish to see that content or does not wish their children to do so. It's easier to teach your model never to produce that kind of content than to teach it to perfectly distinguish whether this user should see that content or not. TV channels are barred from broadcasting this kind of content for similar reasons. Sure, there are alway…

All of this is true but then it's as easy as releasing censored and uncensored versions of the model. Then it's up to users (or parents, in the case of children) to choose the adequate version for each purpose. Just like there are child-friendly movies and adult-only movies, and no one beyond fringe puritan crusaders would say that the latter should outright not exist.

>censored and uncensored

Well here you still have the same problem, since they're not gonna release an actually uncensored version, that tells you how to do awful things (or indeed, that tells you to do them).

So then you'd have censored and less censored, and it would still be a matter of where to draw those lines.

Re: Gemma 3 Technical Report [pdf]

#116

Post by bbminner »

LeoPanthera wrote: post #13

> They are designed to help prevent our models from generating harmful content, i.e., > [...] > Sexually explicit content Dear tech companies. Sexually explicit content is not harmful. Why are you all run by puritans? I don't even want to make edgy porn, I just want to be treated like an adult.

Not all sexually explicit content is harmful in all contexts for sure, but in many contexts it is fairly universally considered harmful (eg content involving minors). Do you have means of distinguishing between the two? Are you suggesting that a company must invests millions into teaching the model where exactly the red line lines so that it can have a conversation close to it but without crossing it? Or you suggest biting the bullet and releasing the model not only capable of generating eg child porn, but also having a >0 chance of randomly discussing it in unrelated contexts? Chance of error is always there, and companies decided that a risk of really bad behavior in benign context overweights the gains. Imho, a decision to not play whack a mole with this land mine is quite rational, esp considering gains vs risks vs costs. Think of it as a cost cutting measure, not as an infringement on free speech. You are free to invest you own money into this problem if you think that's a grave mistake and a missed opportunity. The first project to push the automated generated content moderation against what is considered appropriate in the given context far enough to make it economical for companies to put their guard down could actually be worth a lot if you think there's market for it (eg agents on dating websites? idk, you tell me)

Re: Gemma 3 Technical Report [pdf]

#117

Post by alekandreev »

heinrichf wrote: post #111
alekandreev wrote: post #33

Greetings from the Gemma team! We just got Gemma 3 out of the oven and are super excited to show it to you! Please drop any questions here and we'll answer ASAP. (Opinions our own and not of Google DeepMind.) PS we are hiring: https://boards.greenhouse.io/deepmind/jobs/6590957

I'm comparing Gemma3 12 B ( https://ollama.com/library/gemma3 ; running fully on my 3060 12GB) and Mistral Small 3 24B ( https://ollama.com/library/mistral-small ; 10% offloaded to the CPU). - Gemma3 12B: ~100 t/s on prompt eval; 15 t/s on eval - MistralSmall3 24B: ~500 t/s on prompt eval; 10 t/s on eval Do you know what different in architecture could make the prompt eval (prefill) so much slower on the 2x smaller G…

Thank you for the report! We are working with the Ollama team directly and will look into it.

Re: Gemma 3 Technical Report [pdf]

#118

Post by BoredPositron »

DJHenk wrote: post #105
BoredPositron wrote: post #93

Earlier quoted context omitted.

You don’t know if the censorship is in the model or the system prompt.

That is not relevant to the argument. Censoring limits possibilities. While that sometimes has its uses, the overly puritanical approach American companies generally take degrades the value of their products.

I am talking about an „open“ weight model you are talking about a service. If the service wants to censor that’s fine and on them and their leadership if an „open“ model gets released with censorship it’s not, because it’s just „open, but how my manager likes it“

Re: Gemma 3 Technical Report [pdf]

#119

Post by msp26 »

LeoPanthera wrote: post #13

> They are designed to help prevent our models from generating harmful content, i.e., > [...] > Sexually explicit content Dear tech companies. Sexually explicit content is not harmful. Why are you all run by puritans? I don't even want to make edgy porn, I just want to be treated like an adult.

I want to use a multimodal model for manga translation, analysis, and tagging.

If this gives me the "aschually as a ethical safe harmless assistant I can't ..." spiel on anything mildly mature, that would be very disappointing. I'll run a test with Berserk and see how it goes.

I'm not a big believer in abliteration, it seems to always hurt performance. Safety should be handled by a separate system, no need to cripple the actual LLM.

Re: Gemma 3 Technical Report [pdf]

#120

Post by oezi »

thunkingdeep wrote: post #71
smrq wrote: post #62

Earlier quoted context omitted.

Hard to get more puritanical than "if you disagree with my opinion then you're morally repulsive". Not to mention that your argument implies that all traces of sex ought to be scrubbed from the entire Internet ? And that that conclusion is the only moral one?

Hard to get more perverse than “kids should have access to sexually explicit material at all times in any medium. If that sounds fucked up when I say it like that, consider what assumptions you’re making, because that’s literally YOUR argument here.

Things do not exist on a black and white basis but there are relevant gray scales to be considered:

It is quite different if we talk about removing any sort of text about body parts related to consensual sexual activities or if we try to censor hard pornography or illegal sexual acitivities. I personally find LLM producing sexual content as text rather irrelevant in the same way that you could go to a library or bookstore and buy a romance.

It is also quite different if your definition of kids goes all the way to 18 years. I don't want my kids not to encounter topics surrounding sex until the become legal adults. They absolutely have to learn about it, and be able to develop healthy relationships to their own body and sexuality and have insights that enable them to understand sexuality in others.

I want to protect my kids from harm, but there must be some balance with other aspects as well.

Post reply on HN