Gemma: New Open Models • Hacker News Board

Re: Gemma: New Open Models

#211

Post by spacebanana7 » Wed, Feb 21, 2024, 4:06 PM UTC

w4 wrote: post #142

Parameter counts notwithstanding, it’s an objectively funny outcome that Meta, Microsoft, and Google are all releasing cutting edge open models, while OpenAI keeps theirs closed source.

It's ironic but actually follows their business interests.

Microsoft & google have large cloud divisions that benefit from open models. The lower the cost of AI models, the more they get run and the greater the cloud spend.

Meta is a consumer of AI. They themselves want cheap and effective AI for targeting adverts and building metaverses.

A loose analogy is that both oil producers and car companies want refining to be cheap.

Re: Gemma: New Open Models

#212

Post by DreamGen » Wed, Feb 21, 2024, 4:06 PM UTC

brucethemoose2 wrote: post #122

tosh wrote: post #45

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…

Only 8K context as well, like Mistral. Also, as always, take these benchmarks with a huge grain of salt. Even base model releases are frequently (seemingly) contaminated these days.

Mistral Instruct v0.2 is 32K.

Re: Gemma: New Open Models

#213

Post by dmnsl » Wed, Feb 21, 2024, 4:07 PM UTC

alekandreev wrote: post #3

Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.

Hi, what is the cutoff date ?

Re: Gemma: New Open Models

#214

Post by scarmig » Wed, Feb 21, 2024, 4:08 PM UTC

moffkalast wrote: post #197

scarmig wrote: post #165

Earlier quoted context omitted.

Who cares if it's a PR stunt to improve developer good will? It's still a good thing, and it's now the most open model out there.

How is it more open than Mistral with Apache 2.0? Google wants people to sign a waiver to even download it.

Fair enough; that was more directed at LLaMA and derivatives, which have commercial restrictions.

Re: Gemma: New Open Models

#215

Post by joshelgar » Wed, Feb 21, 2024, 4:13 PM UTC

FergusArgyll wrote: post #75

margorczynski wrote: post #37

Is there a chance we'll get a model without the "aligment" (lobotomization)? There are many examples where answers from Gemini are garbage because of the ideological fine tuning.

You can (and someone will) fine tune it away. There are datasets which are foss you can use on hugging face. Or you can just wait, it'll be done soon...

Could you give an example of these datasets?

Re: Gemma: New Open Models

#216

Post by chown » Wed, Feb 21, 2024, 4:17 PM UTC

If you are looking for a nice chat UI to try out Gemma (and other offline + online models) locally, I'm working on an app [1] that is offline and privacy focused.

I've just added support for Gemma 7B.

[1]: https://msty.app

Re: Gemma: New Open Models

#217

Post by minimaxir » Wed, Feb 21, 2024, 4:17 PM UTC

ericskiff wrote: post #70

Has anyone found the context length for these models yet? So far I haven't seen it mentioned in their write-up or the model card

For posterity, an easy way to find the context length of a LLM hosted on Hugging Face is to look at the max_position_embeddings in the config.json, which shows the 8192 mentioned in another comment. (although in this case you need to sign the agreement first)

Re: Gemma: New Open Models

#218

Post by dougmwne » Wed, Feb 21, 2024, 4:18 PM UTC

willy_k wrote: post #195

dougmwne wrote: post #148

Earlier quoted context omitted.

These are downloadable open models that can be fined tuned. They are the opposite of censored. If you have the motivation, you can bias them however you please.

Is “the opposite of censored” accurate for something that’s default and considerably easier to access mode of operation won’t say many things for sociopolitical reasons? Able to be un censored, sure, but the extent of that is debatable as well.

There is no default and easy access mode. These are raw model weights and only enthusiasts and researchers will download the necessary packages to run it locally. Much more likely is that some popular fine tunes show up on hugging face for more general access.

Re: Gemma: New Open Models

#219

Post by brucethemoose2 » Wed, Feb 21, 2024, 4:19 PM UTC

margorczynski wrote: post #37

Is there a chance we'll get a model without the "aligment" (lobotomization)? There are many examples where answers from Gemini are garbage because of the ideological fine tuning.

Alignment is all but a non issue with open weight base model releases, as they can be finetuned to "de align" them if prompt engineering is not enough.

Re: Gemma: New Open Models

#220

Post by GaggiX » Wed, Feb 21, 2024, 4:21 PM UTC

espadrine wrote: post #158

GaggiX wrote: post #134

Earlier quoted context omitted.

Looking at the config.json of Gemma 7B the feedfoarward hidden size is 8x, not 16x

Huh, indeed, that's what the config.json[0] says; the report[1] indicates “Feedforward hidden dims: 49152”. [0]: https://huggingface.co/google/gemma-7b-it/blob/main/config.j... [1]: https://storage.googleapis.com/deepmind-media/gemma/gemma-re...

I don't see the number 49152 reported in the config.json, what line are you referring to? I just see the intermediate_size of 24576 (so 8x).

EDIT: I didn't read the comment correctly, you have noticed the same thing.