Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma: New Open Models

blog.google

Post reply on HN
Print view View original

451–460 of 543 posts

Re: Gemma: New Open Models

#451

Post by willsmith72 »

verticalscaler wrote: post #420
epistasis wrote: post #352

Earlier quoted context omitted.

Of all the very very very many things that Google models get wrong, not understanding nationality and skin tone distributions seems to be a very weird one to focus on. Why are there three links to this question? And why are people so upset over it? Very odd, seems like it is mostly driven by political rage.

Exactly. It is a wonderful tool, lets focus on classic art instead of nationality: "Depict the Girl with a Pearl Earring" https://pbs.twimg.com/media/GG33L6Ka4AAC-n7?format=jpg&name=... People who are driven by political rage, gaslighters, are really something else, agreed.

Yeah that is just absurd.

Google has been burnt before, e.g. classifying black people as gorillas in 2015, so I can understand their fear when they have so much to lose, but clearly they've gone way too far the other way and are going to have to do a lot to regain people's trust. For now, Gemini is a play toy

https://www.bbc.com/news/technology-33347866.amp

Re: Gemma: New Open Models

#452

Post by alwayslikethis »

simcop2387 wrote: post #362
iandanforth wrote: post #326

Earlier quoted context omitted.

These are all very new licenses that deviate from OSI principles, I think it's fair to call them "unusual".

I think they meant not unusual in this space, not unusual in the sense of open source licensing.

For this sentence to parse, you need to either add or remove a "not".

Re: Gemma: New Open Models

#453

Post by laurentlb »

robswc wrote: post #348
aetherson wrote: post #293

Earlier quoted context omitted.

Were you asking Gemma about this, or Gemini? What were your prompts?

Gemini. I first asked it to tell me about the Heian period (which it got correct) but then it generated images and seemed to craft the rest of the chat to fit that narrative. I mean, just asking it for a "samurai" from the period will give you this: https://g.co/gemini/share/ba324bd98d9b >A non-binary Indigenous American samurai It seems to recognize it's mistakes if you confront it though. The more I mess with it th…

It's funny how they introduced a clear US-centric bias while trying to push for more diversity.

Re: Gemma: New Open Models

#454

Post by generalizations »

threecheese wrote: post #447
philsnow wrote: post #441

Earlier quoted context omitted.

Sounds like it would be interesting to keep track of the model's responses to the same queries over time. > Gemma-2024-Feb, what do you think of the situation in the South China Sea? > > The situation in the South China Sea is complex and multi-faceted, involving a wide range of issues including political conflicts, economic challenges, social changes, and historical tensions. > Gemma-2024-Oct, what do you think of t…

This is a great idea; I wonder if anyone is working on AI censorship monitoring at scale or at all. A secondary model could compare “censorship candidate” prompt results over time to classify how those results changed, and if those changes represent censorship or misinformation.

There's also (I think?) been some research in the direction of figuring out more abstract notions of how models perceive various 'concepts'. I'd be interested in the LLM version of diffs to see where changes have been implemented overall, too.

But really, the trouble is that it's tough to predict ahead of time what kinds of things are likely to be censored in the future; if I were motivated to track this, I'd just make sure to keep a copy of each version of the model in my personal archive for future testing with whatever prompts seem reasonable in the future.

Re: Gemma: New Open Models

#455

Post by generalizations »

reissbaker wrote: post #405
wahnfrieden wrote: post #364

Earlier quoted context omitted.

It says you must make efforts (to a reasonable extent), not that you must give a reason for not making efforts

This is a TOS, meaning their enforcement option is a lawsuit. In court, if you convincingly argue why it would take an unreasonable amount of effort to update, you win. They can't compel you to unreasonable effort as per their own TOS.

This assumes they even know that the model hasn't been updated. Who is this actually intended for? I'd bet it's for companies hosting the model. In those cases, the definition of reasonable effort is a little closer to "it'll break our stuff if we touch it" rather than "oh silly me, I forgot how to spell r-s-y-n-c".

Re: Gemma: New Open Models

#456

Post by Silphendio »

a2128 wrote: post #243
simonw wrote: post #229

The terms of use: https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policy Something that caught my eye in the terms: > Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma. One of the biggest benefits of running your own model is that it can protect you from model updates that break your carefully tested prompts, so I’m not…

This is actually not that unusual. Stable Diffusion's license, CreativeML Open RAIL-M, has the exact same clause: "You shall undertake reasonable efforts to use the latest version of the Model." Obviously updating the model is not very practical when you're using finetuned versions, and people still use old versions of Stable Diffusion. But it does make me fear the possibility that if they ever want to "revoke" every…

It's worth noting that Stable Diffusion XL uses the OpenRAIL++-M License, which removed the update obligation.

Re: Gemma: New Open Models

#457

Post by saintradon »

speedgoose wrote: post #353
brucethemoose2 wrote: post #178

Earlier quoted context omitted.

Miqu was (allegedly) an internal continued pretrain Mistral did as a test, that was leaked as a GGUF. Maybe its just semantics, it is technically a finetune... But to me theres a big difference between expensive "continuation training" (like Solar 10.7B or Mistral 70B) and a much less intense finetuning. The former is almost like releasing a whole new base model. It would be awesome if Mistral did that with their dat…

Arthur Mensch, the Mistral CEO, confirmed the leak. https://twitter.com/arthurmensch/status/1752737462663684344

Also, it led to one of the funniest pr I've seen in a while

https://huggingface.co/miqudev/miqu-1-70b/discussions/10

Re: Gemma: New Open Models

#458

Post by not2b »

lordswork wrote: post #108
artninja1988 wrote: post #61

Earlier quoted context omitted.

The statement on you not being able to use LLaMA 2 to benchmark is also false and highly misleading see https://x.com/BlancheMinerva/status/1760302091166241163?s=20

If, on the Llama 2 version release date, the monthly active users [...] is greater than 700 million monthly active users [...] you are not authorized to exercise any of the rights under this Agreement I would guess this is Google being careful to not be burned by this lame clause in the Llama 2 license.

It's aimed directly at them (and OpenAI and Microsoft) so they have to honor it if they don't want a legal battle. But there's nothing stopping others from doing benchmarking.

Re: Gemma: New Open Models

#459

Post by etiam »

OJFord wrote: post #59
benpacker wrote: post #41

Earlier quoted context omitted.

Yes - they are open weights and open inference code, which means they can be integrated into Ollama. They are not “open training” (either in the training code or training data sense), so they are not reproducible, which some have suggested ought to be a component of the definition of open models.

It really should shouldn't it? I'm quite ML-naïve, but surely providing the model without 'training code or training data' is just like providing a self-hostable binary without the source code? Nobody calls that open source, it's not even source available.

Yes, and there has been some discussion of that

Meta’s LLaMa 2 license is not Open Source https://news.ycombinator.com/item?id=36820122

Re: Gemma: New Open Models

#460

Post by alekandreev »

CuriouslyC wrote: post #98
alekandreev wrote: post #3

Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.

It's cool that you guys are able to release open stuff, that must be a nice change from the modus operandi at goog. I'll have to double check but it looks like phi-2 beats your performance in some cases while being smaller, I'm guessing the value proposition of these models is being small and good while also having more knowledge baked in?

We deeply respect the Phi team and all other teams in the open model space. You’ll find that different models have different strengths and not all can be quantified with existing public evals. Take them for a spin and see what works for you.
Post reply on HN