Viewing profile — alekandreev

Yes we have measured the tradeoff. We don't see a drop of perplexity in English when introducing multilingual, and there is a slight drop in some English language-specific evals (~…

We never train at 128k, only 32k, changing the scaling factor at the end. We wanted the long context recipe to be friendly for finetuning, and training at 128k is a bit of a pain w…

We recommend using user for the system prompt as well.

That's a very interesting area, but nothing we can announce today.

Hey, Gemma engineer here. Can you please share reports on the type of prompts and the implementation you used?

Thank you for the report! We are working with the Ollama team directly and will look into it.

Thank you for the feedback! This is why we are so excited to push more and more on small models for both low end and high end smartphones!

That's an idea we've thought about. However, we think the open source community has already created a very impressive set of language or region-specific finetunes [1] [2]. Also the…

Picking model sizes is not an exact science. We look for sizes that will fit quantized on different categories on devices (e.g., low-end and high-end smartphone, laptops and 16GB G…

Greetings from the Gemma team! We just got Gemma 3 out of the oven and are super excited to show it to you! Please drop any questions here and we'll answer ASAP. (Opinions our own …

I think it makes sense to compare models trained with the same recipe on token count - usually more tokens will give you a better model. However, I wouldn't draw conclusions about …

To quote Ludovic Peran, our amazing safety lead: Literature has identified self-proliferation as dangerous capability of models, and details about how to define it and example of f…

Happy to pass on any feedback to our Google Cloud friends. :)

This is mostly about inference speed, while maintaining long context performance.

In addition to the HF links shared by sibling comments, the 2B will be released soon.

The terms of use remain the same as Gemma 1 - https://ai.google.dev/gemma/terms .

Your training input has the shape of (sequence length x batch size). If a lot of your samples are shorter than sequence length, as is usually the case, you will have a lot of paddi…

Hello (again) from the Gemma team! We are quite excited to push this release out and happy to answer any questions! Opinions are our own and not of Google DeepMind.

As a fellow Bulgarian from the 80s and 90s myself, and now a part of the Gemma team, I’d say Austin, Jan, and team very much live up to the ethos of hackers I'd meet on BBSes back …

Sorry, doing our best here :)

We have many exciting things planned that we can't reveal just yet :)

September 2023.