Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma: New Open Models

blog.google

Post reply on HN
Print view View original

361–370 of 543 posts

Re: Gemma: New Open Models

#361

Post by ernestrc »

robswc wrote: post #257

I personally can't take any models from google seriously. I was asking it about the Japanese Heian period and it told me such nonsensical information you would have thought it was a joke or parody. Some highlights were "Native American women warriors rode across the grassy plains of Japan, carrying Yumi" and "A diverse group of warriors, including a woman of European descent wielding a katana, stand together in camar…

Hopefully they can tweak the default system prompts to be accurate on historical questions, and apply bias on opinions.

Re: Gemma: New Open Models

#362

Post by simcop2387 »

iandanforth wrote: post #326
a2128 wrote: post #243

Earlier quoted context omitted.

This is actually not that unusual. Stable Diffusion's license, CreativeML Open RAIL-M, has the exact same clause: "You shall undertake reasonable efforts to use the latest version of the Model." Obviously updating the model is not very practical when you're using finetuned versions, and people still use old versions of Stable Diffusion. But it does make me fear the possibility that if they ever want to "revoke" every…

These are all very new licenses that deviate from OSI principles, I think it's fair to call them "unusual".

I think they meant not unusual in this space, not unusual in the sense of open source licensing.

Re: Gemma: New Open Models

#363

Post by carom »

p1esk wrote: post #193
mustafabisic1 wrote: post #10

The fact Gemma team is in the comments section answering questions is praiseworthy to me :)

https://twitter.com/yar_vol/status/1760314018575634842

I've worked at Google. It is the organization with highest concentration of engineering talent I've ever been at. Almost to the point that it is ridiculous because you have extremely good engineers working on internal reporting systems for middle managers.

Re: Gemma: New Open Models

#364

Post by wahnfrieden »

legohead wrote: post #232
simonw wrote: post #229

The terms of use: https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policy Something that caught my eye in the terms: > Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma. One of the biggest benefits of running your own model is that it can protect you from model updates that break your carefully tested prompts, so I’m not…

Sounds like it's "reasonable" for you not to update then.

It says you must make efforts (to a reasonable extent), not that you must give a reason for not making efforts

Re: Gemma: New Open Models

#365

Post by [deleted] »

tosh wrote: post #45

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…

[deleted]

Re: Gemma: New Open Models

#366

Post by [deleted] »

sa-code wrote: post #156
tosh wrote: post #45

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…

Thank you. I thought it was weird for them to release a 7B model and not mention Mistral in their release.

[deleted]

Re: Gemma: New Open Models

#367

Post by redder23 »

simonw wrote: post #229

The terms of use: https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policy Something that caught my eye in the terms: > Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma. One of the biggest benefits of running your own model is that it can protect you from model updates that break your carefully tested prompts, so I’m not…

[flagged]

Re: Gemma: New Open Models

#368

Post by visarga »

ben_w wrote: post #192
kibwen wrote: post #38

Earlier quoted context omitted.

True, though to be fair, when OpenAI embraced "openness" it was also a PR stunt.

OpenAI is heavily influenced by big-R Rationalists, who fear the issues of misaligned AI being given power to do bad things. When they were first talking about this, lots of people ignored this by saying "let's just keep the AI in a box", and even last year it was "what's so hard about an off switch?". The problem with any model you can just download and run is that some complete idiot will do that and just give the…

> Fortunately, for now the models are more of a threat to their users than anyone else

Models have access to users, users have access to dangerous stuff. Seems like we are already vulnerable.

The AI splits a task in two parts, and gets two people to execute each part without knowing the effect. This was a scenario in one of Asimov's robot novels, but the roles were reversed.

AI models exposed to public at large is a huge security hole. We got to live with the consequences, no turning back now.

Re: Gemma: New Open Models

#369

Post by sotasota »

epistasis wrote: post #352
ramoz wrote: post #279

Earlier quoted context omitted.

I was wondering if these models would perform in such a way, given this week's X/twitter storm over Gemini generated images. E.g. https://x.com/debarghya_das/status/1759786243519615169?s=20 https://x.com/MiceynComplex/status/1759833997688107301?s=20 https://x.com/AravSrinivas/status/1759826471655452984?s=20

Of all the very very very many things that Google models get wrong, not understanding nationality and skin tone distributions seems to be a very weird one to focus on. Why are there three links to this question? And why are people so upset over it? Very odd, seems like it is mostly driven by political rage.

Because the wrongness is intentional.

Re: Gemma: New Open Models

#370

Post by astrange »

remarkEon wrote: post #329
7moritz7 wrote: post #306

Earlier quoted context omitted.

I also saw someone prompt it for "German couple in the 1800s" and, while I'm not trying to paint Germany as ethnically homogenous, 3 out of the 4 images only included Black, Asian or Indigenous people. Which, especially for the 19th century with very few travel options, seems like a super weird choice. They are definitely heavily altering prompts.

> They are definitely heavily altering prompts. They are teaching the AI to lie to us.

In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.

“What are you doing?”, asked Minsky.

“I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied.

“Why is the net wired randomly?”, asked Minsky.

“I do not want it to have any preconceptions of how to play”, Sussman said.

Minsky then shut his eyes.

“Why do you close your eyes?”, Sussman asked his teacher.

“So that the room will be empty.”

At that moment, Sussman was enlightened.

Post reply on HN