Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma: New Open Models

blog.google

Post reply on HN
Print view View original

151–160 of 543 posts

Re: Gemma: New Open Models

#151

Post by infecto »

idiotsecant wrote: post #96
infecto wrote: post #88

Earlier quoted context omitted.

I have a different take, Google releases a lot but is also a massive company and tools like Chromium serve to increase their stock price so they can hit their quarterly estimates.

In what way does chromium increase stock price? In what way does stock price influence quarterly estimates? Are we playing business words mad libs?

I don't know why people like yourself respond with such derisive commentary instead of simply asking the constructive question.

Initially? It fueled dethroning MSFT and help gain marketshare for Chrome. On a go-forward basis it allows Google to project massive weight in standards. In extension to its use with Chrome, Chrome is a significant knob for ad revenue that they utilize to help meet expectations. That knob only exists because of its market share.

Re: Gemma: New Open Models

#152

Post by surajrmal »

makestuff wrote: post #115
brainless wrote: post #58

Earlier quoted context omitted.

This article states quite an impressive list of open source tools that Google has released for years in the past. This is no surprise coming from* them. Google has released some large pieces of source in other domains as well, Chromium comes to mind, which probably impacts most Internet users directly. The question is not about Google but about OpenAI.

Google also has released Guice/Dagger for Java dependency injection. Angular never really took off, but guice/dagger are widely used. Also I am pretty impressed with Flutter as an alternative to react native.

Angular was incredibly popular for a long time and still is. Usage is shifting down over time but a lot of notable websites still use it.

Re: Gemma: New Open Models

#153

Post by FergusArgyll »

tosh wrote: post #45

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…

the real gold will be when this gets finetuned. (maybe by mistral...)

Re: Gemma: New Open Models

#154

Post by calebkaiser »

impulser_ wrote: post #27

Go back 5 years and ask anyone on this site what companies do you think will be the most open about AI in the future OpenAI, Meta, or Google. I bet 10/10 people would pick OpenAI. Now today Meta and Google, both trillion dollars companies, are releasing very powerful open models with the ability to be used commercially. Ironic.

Since the release of GPT-2 (it was initially "too dangerous" to release the weights), I think most people in the industry have assumed that OpenAI does not see open sourcing their models as a strategic advantage.

Re: Gemma: New Open Models

#155

Post by artninja1988 »

trisfromgoogle wrote: post #131
artninja1988 wrote: post #46

Earlier quoted context omitted.

The synde remarks at metas llama license that doesn't allow companies with 700 million monthly active users to use it, while this model also doesn't have a really 'open' license itself and also this paragraph: >As the ecosystem evolves, we urge the wider AI community to move beyond simplistic ’open vs. closed’ debates, and avoid either exaggerating or minimising potential harms, as we believe a nuanced, collaborative…

Quick question -- can you tell me where you got that quote? It's not in the main blog or any of the launch communications that I can see.

The quote is from the technical report

https://storage.googleapis.com/deepmind-media/gemma/gemma-re...

Re: Gemma: New Open Models

#156

Post by sa-code »

tosh wrote: post #45

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…

Thank you. I thought it was weird for them to release a 7B model and not mention Mistral in their release.

Re: Gemma: New Open Models

#157

Post by brucethemoose2 »

FergusArgyll wrote: post #153
tosh wrote: post #45

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…

the real gold will be when this gets finetuned. (maybe by mistral...)

TBH the community has largely outrun Mistral's own finetuning. The 7B model in particular is such a popular target because its so practical to train.

Re: Gemma: New Open Models

#158

Post by espadrine »

GaggiX wrote: post #134
espadrine wrote: post #29

I notice a few divergences to common models: - The feedforward hidden size is 16x the d_model, unlike most models which are typically 4x; - The vocabulary size is 10x (256K vs. Mistral’s 32K); - The training token count is tripled (6T vs. Llama2's 2T) Apart from that, it uses the classic transformer variations: MQA, RoPE, RMSNorm. How big was the batch size that it could be trained so fast? https://huggingface.co/mis…

Looking at the config.json of Gemma 7B the feedfoarward hidden size is 8x, not 16x

Huh, indeed, that's what the config.json[0] says; the report[1] indicates “Feedforward hidden dims: 49152”.

[0]:https://huggingface.co/google/gemma-7b-it/blob/main/config.j...

[1]: https://storage.googleapis.com/deepmind-media/gemma/gemma-re...

Re: Gemma: New Open Models

#159

Post by lawxls »

tosh wrote: post #45

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…

Honestly, this is more of a PR stunt to advertise the Google Dev ecosystem than a contribution to open-source. I'm not complaining, just calling it what it is.

Barely an improvement over the 5-month-old Mistral model, with the same context length of 8k. And this is a release after their announcement of Gemini Pro 1.5, which had an exponential increase in context length.

Re: Gemma: New Open Models

#160

Post by alextheparrot »

idiotsecant wrote: post #96
infecto wrote: post #88

Earlier quoted context omitted.

I have a different take, Google releases a lot but is also a massive company and tools like Chromium serve to increase their stock price so they can hit their quarterly estimates.

In what way does chromium increase stock price? In what way does stock price influence quarterly estimates? Are we playing business words mad libs?

> “Our best shot at making the quarter is if we get an injection of at least [redacted]% , queries ASAP from Chrome.” (Google Exec)

Isn’t there a whole anti-trust case going on around this?

[0] https://www.nytimes.com/interactive/2023/10/24/business/goog...

Post reply on HN