Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
Hi, what is the cutoff date ?
All it will tell me is mid-2018.
Gemma: New Open Models
261–270 of 543 posts
Re: Gemma: New Open Models
#262Earlier quoted context omitted.
Who cares if it's a PR stunt to improve developer good will? It's still a good thing, and it's now the most open model out there.
How exactly is it the "most open model" ? It's more like a masterclass in corporate doublespeak. Google’s "transparency" is as clear as mud, with pretraining details thinner than their privacy protections. Diving into Google’s tech means auctioning off your privacy (and your users' privacy) to the highest bidder. Their "open source" embrace is more of a chokehold, with their tech biases and monopolistic strategies ba…
You said a lot of nothing without actually saying specifically what the problem is with the recent license.
Maybe the license is fine for almost all usecases and the limitations are small?
For example, you complained about metas license, but basically everyone uses those models and is completely ignoring it. The weights are out there, and nobody cares what the fine print says.
Maybe if you are a FAANG, company, meta might sue. But everyone else is getting away with it completely.
Re: Gemma: New Open Models
#263Earlier quoted context omitted.
I have a different take, Google releases a lot but is also a massive company and tools like Chromium serve to increase their stock price so they can hit their quarterly estimates.
In what way does chromium increase stock price? In what way does stock price influence quarterly estimates? Are we playing business words mad libs?
Chromium is open source because its roots are as a fork of WebKit (Safari). Which itself was open source because it was a fork of KHTML from KDE.
Google stood on the shoulders of others to get out a browser that drives 80% of their desktop ad revenue.
How does that not affect GOOG?
Re: Gemma: New Open Models
#264The scariest difference between OpenAI and Google right now is: Ask Gemini who owns the code it writes, and it'll confidently say that Google does. Ask OpenAI, and it'll say that you do. It's that easy to choose which one is the better decision.
Considering the nuanced nature of copyrighting AI outputs, it isn't clear that either answer is correct.
Re: Gemma: New Open Models
#265The terms of use: https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policy Something that caught my eye in the terms: > Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma. One of the biggest benefits of running your own model is that it can protect you from model updates that break your carefully tested prompts, so I’m not…
Huh. I wonder why is that a part of the terms. I feel like that's more of a support concern.
[deleted]
Re: Gemma: New Open Models
#266Earlier quoted context omitted.
Does this model also thinks german were black 200 years ago ? Or is afraid to answer basic stuff ? because if this is the case no one will care about that model.
I don't know anything about these twitter accounts so I don't know how credible they are, but here are some examples for your downvoters that I'm guessing just think you're just trolling or grossly exaggerating: https://twitter.com/aginnt/status/1760159436323123632 https://twitter.com/Black_Pilled/status/1760198299443966382
Yea. Just ask it anything about historical people/cultures and it will seemingly lobotomize itself.
I asked it about early Japan and it talked about how European women used Katanas and how Native Americans rode across the grassy plains carrying traditional Japanese weapons. Pure made up nonsense that not even primitive models would get wrong. Not sure what they did to it. I asked it why it assumed Native Americans were in Japan in the 1100s and it said:
> I assumed [...] various ethnicities, including Indigenous American, due to the diversity present in Japan throughout history. However, this overlooked [...] I focused on providing diverse representations without adequately considering the specific historical context.
How am I supposed to take this seriously? Especially on topics I'm unfamiliar with?
Re: Gemma: New Open Models
#267Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
Is there any truth behind this claim that folks who worked on Gemma have left Google? https://x.com/yar_vol/status/1760314018575634842
Them: here to answer questions
Question
Them: :O
Re: Gemma: New Open Models
#268Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…
According to their paper, average of standard task of Mistral is 54.0 and for Gemma it's 56.4, so 4.4% relative better. Not as big as you would expect for the company which invented transformers and probably has 2-3 order more compute for training it vs few month old French startup.
Also for note on their human evaluations, Gemma 7B IT has a 51.7% win rate against Mistral v0.2 7B Instruct.
Re: Gemma: New Open Models
#269Are these any good? I have been trying the non pro version of Gemini, and that seems awful at code generation. I am more keen on getting access to the best model and I would pay for it if I wasn't already paying for ChatGPT 4.
I often talk with GPT4 on road trips about topics I'm interested in. Its great for passing the time.
I tried the same thing with Gemini and its full of nonsense. I was talking with it about the "Heian period" of Japan and it made up all sorts of stuff but you really only could tell because it was so ridiculous. Talked about European women and Native Americans roaming around the famous grassy plains of japan wielding katana and traditional weaponry... in the 1100s.
No such issue with GPT4.
I haven't tried it with code though, since I already have co-pilot. Really hard to trust anything it says after it started making stuff up about such a simple time period.
Re: Gemma: New Open Models
#270I really don't get why there is this obsession with safe "Responsible Generative AI".
I mean it writes some bad words, or bad pics, a human can do that without help as well.
The good thing about dangerous knowledge and generative AI is that you're never sure haha, you'd be a fool to ask GPT to make a bomb. I mean it would probably be safe, since it will make up half of the steps.