Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma: New Open Models

blog.google

Post reply on HN
Print view View original

141–150 of 543 posts

Re: Gemma: New Open Models

#141

Post by vertis »

ekianjo wrote: post #123
DJHenk wrote: post #121

Earlier quoted context omitted.

> Ironic. Not at all. When you're the underdog, it makes perfect sense to be open because you can profit from the work of the community and gain market share. Only after establishing some kind of dominance or monopoly it makes sense (profit wise) to switch to closed technology. OpenAI was open, but is now the leader and closed up. Meta and Google need to play catch up, so they are open.

> OpenAI was open When is the last time they released something in the open?

I think that's the point, they released GPT2 openly, but as soon as they had something commercially viable they became ClosedAI.

Re: Gemma: New Open Models

#142

Post by w4 »

Parameter counts notwithstanding, it’s an objectively funny outcome that Meta, Microsoft, and Google are all releasing cutting edge open models, while OpenAI keeps theirs closed source.

Re: Gemma: New Open Models

#144

Post by dkjaudyeqooe »

DJHenk wrote: post #121
impulser_ wrote: post #27

Go back 5 years and ask anyone on this site what companies do you think will be the most open about AI in the future OpenAI, Meta, or Google. I bet 10/10 people would pick OpenAI. Now today Meta and Google, both trillion dollars companies, are releasing very powerful open models with the ability to be used commercially. Ironic.

> Ironic. Not at all. When you're the underdog, it makes perfect sense to be open because you can profit from the work of the community and gain market share. Only after establishing some kind of dominance or monopoly it makes sense (profit wise) to switch to closed technology. OpenAI was open, but is now the leader and closed up. Meta and Google need to play catch up, so they are open.

> Not at all. When you're the underdog, it makes perfect sense to be open because you can profit from the work of the community and gain market share. Only after establishing some kind of dominance or monopoly it makes sense (profit wise) to switch to closed technology.

That is purely the language of commerce. OpenAI was supposed to be a public benefit organisation, but it acts like a garden variety evil corp.

Even garden variety evil corps spend decades benefitting society with good products and services before they become big and greedy, but OpenAI skipped all that and just cut to the chase. It saw an opening with the insane hype around ChatGPT and just grabbed all it could as fast as it could.

I have a special contempt for OpenAI on that basis.

Re: Gemma: New Open Models

#145

Post by austinvhuang »

nalzok wrote: post #28

Congratulations on the release! How can we download the model and run inference locally?

You can download the model checkpoints from kaggle https://www.kaggle.com/models/google/gemma and huggingface https://huggingface.co/blog/gemma

Besides the python implementations, we also implemented a standalone C++ implementation that runs locally with just CPU simd https://github.com/google/gemma.cpp

Re: Gemma: New Open Models

#147

Post by dkjaudyeqooe »

ta8645 wrote: post #65
kibwen wrote: post #38

Earlier quoted context omitted.

True, though to be fair, when OpenAI embraced "openness" it was also a PR stunt.

My impression is that OpenAI was founded by true believers, with the best intentions; whose hopes were ultimately sidelined in the inexorable crush of business and finance.

> OpenAI was founded by true believers, with the best intentions

who were easily bought off.

Re: Gemma: New Open Models

#149

Post by jcuenod »

tosh wrote: post #45

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…

Came here to post the same thing for Phi-2:

  +-------------+----------+-------------+
  | Benchmark   | Gemma 2B | Phi-2 2.7B  |
  +-------------+----------+-------------+
  | MMLU        |   42.3   |     56.7    |
  | MBPP        |   29.2   |     59.1    |
  | BoolQ       |   69.4   |     83.3    |
  +-------------+----------+-------------+
[0] https://www.kaggle.com/models/google/gemma

[1] https://www.microsoft.com/en-us/research/blog/phi-2-the-surp...

Re: Gemma: New Open Models

#150

Post by tosh »

brucethemoose2 wrote: post #122
tosh wrote: post #45

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…

Only 8K context as well, like Mistral. Also, as always, take these benchmarks with a huge grain of salt. Even base model releases are frequently (seemingly) contaminated these days.

Agree: will be interesting how Gemma does on ChatBot Arena
Post reply on HN