Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma: New Open Models

blog.google

Post reply on HN
Print view View original

21–30 of 543 posts

Re: Gemma: New Open Models

#21

Post by artninja1988 »

alekandreev wrote: post #3

Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.

I find the snyde remarks around open source in the paper and announcement rather off putting.

As the ecosystem evolves, we urge the corporate AI community to move beyond demanding to be taken seriously as a player in open source for models that are not actually open, and avoid preaching with a PR statement that can be interpreted as uniformed at best or malicious at worst.

Re: Gemma: New Open Models

#23

Post by wrexx0r »

trisfromgoogle wrote: post #12
h1t35h wrote: post #8

Earlier quoted context omitted.

It seems you have exposed the internal debugging tool link in the blog post. You may want to do something about it.

Ah, I see -- the link is wrong, thank you for flagging! Fixing now.

The link in the Debugging section redirects to a Google SSO login page

Re: Gemma: New Open Models

#25

Post by vanderboyd »

The 2B model seems underwhelming. For instance, compared to the recent StableLM2 1.6B model that is slightly smaller and probably wastes some "English metric points" by being multilingual.

The latter (and other similar open models) seem to do similarly well in benchmarks (much better in Math?) with way less fancy stuff. For instance, public data and no secretive filtering with pre trained models or synthetic data.

My take is that using the vanilla approaches take you really far. And many of the latest tricks and hours-of-work buy you little... Will be interesting to see how this plays out, especially for the open source community.

Re: Gemma: New Open Models

#26

Post by ErneX »

pama wrote: post #14
alekandreev wrote: post #3

Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.

Will these soon be available on lmsys for human comparison against other models? Can they run with llama.cpp?

Yes to llama.cpp

https://twitter.com/ggerganov/status/1760293079313973408

Re: Gemma: New Open Models

#27

Post by impulser_ »

Go back 5 years and ask anyone on this site what companies do you think will be the most open about AI in the future OpenAI, Meta, or Google. I bet 10/10 people would pick OpenAI. Now today Meta and Google, both trillion dollars companies, are releasing very powerful open models with the ability to be used commercially.

Ironic.

Re: Gemma: New Open Models

#29

Post by espadrine »

I notice a few divergences to common models:

- The feedforward hidden size is 16x the d_model, unlike most models which are typically 4x;

- The vocabulary size is 10x (256K vs. Mistral’s 32K);

- The training token count is tripled (6T vs. Llama2's 2T)

Apart from that, it uses the classic transformer variations: MQA, RoPE, RMSNorm.

How big was the batch size that it could be trained so fast?

https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/bl...

Re: Gemma: New Open Models

#30

Post by silentsanctuary »

artninja1988 wrote: post #21
alekandreev wrote: post #3

Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.

I find the snyde remarks around open source in the paper and announcement rather off putting. As the ecosystem evolves, we urge the corporate AI community to move beyond demanding to be taken seriously as a player in open source for models that are not actually open, and avoid preaching with a PR statement that can be interpreted as uniformed at best or malicious at worst.

Which remarks are you referring to?
Post reply on HN