Has anyone found the context length for these models yet? So far I haven't seen it mentioned in their write-up or the model card
The context length for these models is 8192 tokens.
Gemma: New Open Models
91–100 of 543 posts
Re: Gemma: New Open Models
#92Great! Google is now participating in the AI race to zero with Meta, as predicted that $0 free AI models would eventually catch up against cloud-based ones. You would not want to be in the middle of this as there is no moat around this at all. Not even OpenAI.
LLM is the dumb pipe but so far ChatGPT is the most successful generative AI product. It remains to be seen. OpenAI’s models are barely leading Gemini Ultra now, but as chat product it is still miles ahead of the Gemini interface.
The main problem of Gemini 1.5 is that you cannot access it at all as a user :|
Re: Gemma: New Open Models
#93Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
Are there any plans for releasing the datasets used?
This would be really interesting in my opinion, but we are not releasing datasets at this time. See the C4 dataset for an earlier open dataset from Google.
Re: Gemma: New Open Models
#94Earlier quoted context omitted.
Thank you! You can get started downloading the model and running inference on Kaggle: https://www.kaggle.com/models/google/gemma ; for a full list of ways to interact with the model, you can check out https://ai.google.dev/gemma .
FYI the ; broke the link, but I found it easily anyway.
Good catch - just corrected. Thanks!
Re: Gemma: New Open Models
#95Earlier quoted context omitted.
It really should shouldn't it? I'm quite ML-naïve, but surely providing the model without 'training code or training data' is just like providing a self-hostable binary without the source code? Nobody calls that open source, it's not even source available.
That’s why they’re called open as in free to use how you wish, not open source where the source of the training is also provided.
But my point is there's no analogy for that that we call open? It's like self-hostable, or free (as in beer).
Re: Gemma: New Open Models
#96Earlier quoted context omitted.
This article states quite an impressive list of open source tools that Google has released for years in the past. This is no surprise coming from* them. Google has released some large pieces of source in other domains as well, Chromium comes to mind, which probably impacts most Internet users directly. The question is not about Google but about OpenAI.
I have a different take, Google releases a lot but is also a massive company and tools like Chromium serve to increase their stock price so they can hit their quarterly estimates.
In what way does chromium increase stock price? In what way does stock price influence quarterly estimates? Are we playing business words mad libs?
Re: Gemma: New Open Models
#97Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
Can the Gemma models be downloaded to run locally, like open-source models Llama2, Mistral, etc ? Or is your definition of "open" different?
Their definition of "open" is "not open", i.e. you're only allowed to use Gemma in "non-harmful" way.
We all know that Google thinks that saying that 1800s English kings were white is "harmful".
Re: Gemma: New Open Models
#98Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
It's cool that you guys are able to release open stuff, that must be a nice change from the modus operandi at goog. I'll have to double check but it looks like phi-2 beats your performance in some cases while being smaller, I'm guessing the value proposition of these models is being small and good while also having more knowledge baked in?
Re: Gemma: New Open Models
#99Earlier quoted context omitted.
This article states quite an impressive list of open source tools that Google has released for years in the past. This is no surprise coming from* them. Google has released some large pieces of source in other domains as well, Chromium comes to mind, which probably impacts most Internet users directly. The question is not about Google but about OpenAI.
I have a different take, Google releases a lot but is also a massive company and tools like Chromium serve to increase their stock price so they can hit their quarterly estimates.
It was not at all done for the good of the web, it was a mere logical calculation; it was cheaper to develop Chromium, than to pay 4B USD in search royalties to Microsoft Internet Explorer, and would give more control and long-term safety to Google.
Re: Gemma: New Open Models
#100What is the context window?
The context length for these models is 8192 tokens.