Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
Hi, what is the cutoff date ?
September 2023.
Gemma: New Open Models
461–470 of 543 posts
Re: Gemma: New Open Models
#462Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
Will there be Gemma-vision models or multimodal Gemma models?
We have many exciting things planned that we can't reveal just yet :)
Re: Gemma: New Open Models
#463Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
> We are really excited to answer any questions you may have about our models. I cannot count how many times I've seen similar posts on HN, followed by tens of questions from other users, three of which actually get answered by the OP. This one seems to be no exception so far.
Sorry, doing our best here :)
Re: Gemma: New Open Models
#464Earlier quoted context omitted.
Exactly. It is a wonderful tool, lets focus on classic art instead of nationality: "Depict the Girl with a Pearl Earring" https://pbs.twimg.com/media/GG33L6Ka4AAC-n7?format=jpg&name=... People who are driven by political rage, gaslighters, are really something else, agreed.
Yeah that is just absurd. Google has been burnt before, e.g. classifying black people as gorillas in 2015, so I can understand their fear when they have so much to lose, but clearly they've gone way too far the other way and are going to have to do a lot to regain people's trust. For now, Gemini is a play toy https://www.bbc.com/news/technology-33347866.amp
Completely unrelated, enough excuses. This is not some sort of mistake or overcorrection, it is by explicit overt design.
These cowards will never regain my trust. I won't hire or work with or for googlers or any DEI people ever.
Re: Gemma: New Open Models
#465The terms of use: https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policy Something that caught my eye in the terms: > Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma. One of the biggest benefits of running your own model is that it can protect you from model updates that break your carefully tested prompts, so I’m not…
I appreciated this post clarifying the distinction between "open model" and "open source":
https://opensource.googleblog.com/2024/02/building-open-mode...
I'm not sure how to feel about the restrictions. "No porn" feels prudish, particularly for this millennium. I tend to err on the side of freedom in intellectual/political matters; however, the others seem fairly reasonable as far as restrictions go.
Re: Gemma: New Open Models
#466Earlier quoted context omitted.
the real gold will be when this gets finetuned. (maybe by mistral...)
how does one finetune llama (or any other LLM) using mistral? is the flow like this? - take small dataset - generate bigger dataset using mistral (how this is this done?) - run LoRA to fine tune gemma extended dataset.
I should have said "run LoRA or your favorite fine-tuning technique to produce your fine-tuned llama."
Re: Gemma: New Open Models
#467Earlier quoted context omitted.
You can download the model checkpoints from kaggle https://www.kaggle.com/models/google/gemma and huggingface https://huggingface.co/blog/gemma Besides the python implementations, we also implemented a standalone C++ implementation that runs locally with just CPU simd https://github.com/google/gemma.cpp
Are there any cool highlights you can give us about gemma.cpp? Does it have any technical advantages over llama.cpp? It looks like it introduces its own quantization format, is there a speed or accuracy gain over llama.cpp's 8-bit quantization?
Hi, I devised the 4.5 (NUQ) and 8-bit (SFP) compression schemes. These are prototypes that enabled reasonable inference speed without any fine-tuning, and compression/quantization running in a matter of seconds on a CPU.
We do not yet have full evals because the harness was added very recently, but observe that the non-uniform '4-bit' (plus tables, so 4.5) has twice the SNR of size-matched int4 with per-block scales.
One advantage that gemma.cpp offers is that the code is quite compact due to C++ and the single portable SIMD implementation (as opposed to SSE4, AVX2, NEON). We were able to integrate the new quantization quite easily, and further improvements are planned.
Re: Gemma: New Open Models
#468It looks like it's pretty resistant to quantization. ollama 4bit 7B doesn't work very well, but the 16bit 2B does
That's useful to know. My experiments with the 4bit 7B currently tagged for use on ollama are not going well at all. Lots of refusals and junk. Downloading 7b-instruct-fp16 now! :-) (Update: Yes, much better, though much slower too, of course.)
FYI we have an experimental 4.5-bit format (NUQ) in gemma.cpp, our C++ inference engine, which from my testing works reasonably well.
Re: Gemma: New Open Models
#469Earlier quoted context omitted.
It would be great to understand what you mean by this -- we have a deep love for open source and the open developer ecosystem. Our open source team also released a blog today describing the rationale and approach for open models and continuing AI releases in the open ecosystem: https://opensource.googleblog.com/2024/02/building-open-mode... Thoughts and feedback welcome, as always.
The statement on you not being able to use LLaMA 2 to benchmark is also false and highly misleading see https://x.com/BlancheMinerva/status/1760302091166241163?s=20
For the reference of people seeing this now: The tweet that person linked has now been deleted and the scientist who tweeted it has acknowledged they were wrong and retracted their claim, as all good scientists should.
Re: Gemma: New Open Models
#470Earlier quoted context omitted.
Because the wrongness is intentional.
Exactly. Sure this particular example is driven by political rage, but the underlying issue is that the maintainers of these models are altering them to conform to an agenda. It's not even surprising that people choose to focus on the political rage aspect of it, because that same political rage is the source of the agenda in the first place. It's a concerning precedent to set, because what other non-political modifi…
Well, every model is altered to conform to an agenda. You will train it on data, which you have personally picked (and is therefore subject to your own bias), and you'll guide its training to match the goal you wish to achieve with the model. If you were doing the training, your own agenda would come into play. Google's agenda is to make something very general that works for everyone.
So if you're trying to be as unbiased as humanly possible, you might say, just use the raw datasets that exist in the world. But we live in a world where the datasets themselves are often biased.
Bias in ML and other types of models is well-documented, and can cause very real repercussions. Poor representation in datasets can cause groups to be unfairly disadvantaged when an insurance premium or mortgage is calculated, for example. It can also mean your phone's ML photography system doesn't expose certain skin colors very well.
Even if it was trained with a statistically representative dataset (e.g. about 2/3 of the US is white), you want your model to work for ALL your customers, not just 2/3 of them. Since ML has a lot to do with statistics, your trained model will see "most of this dataset is white" and the results will reflect that. So it is 100% necessary to make adjustments if you want your model to work accurately for everyone, and not just the dominant population in the dataset.
Even if we aren't using these models for much yet, a racist AI model would seriously harm how people trust and rely on these models. As a result, training models to avoid bias is 100% an important part of the agenda, even when the agenda is just creating a model that works well for everyone.
Obviously, that's gone off the rails a bit with these examples, but it is a real problem nonetheless. (And training a model to understand the difference between our modern world and what things were like historically is a complex problem, I'm sure!)