Earlier quoted context omitted.
About 5 months until we see widespread local LLMs, thanks to Apple.
Apple needs to be known as an AI leader first.
Why?
Gemma: New Open Models
191–200 of 543 posts
Re: Gemma: New Open Models
#192Earlier quoted context omitted.
Not surprising, just like when MS went to shit, and then they start to embrace 'open source'. Seems like PR stunt. And when it comes to LLM there is millions of dollar barrier to entry to train the model, so it is ok to open up their embedding etc. Today big corp A will open up a little to court the developers, and tomorrow when it gains dominance it will close up, and corp B open up a little.
True, though to be fair, when OpenAI embraced "openness" it was also a PR stunt.
OpenAI is heavily influenced by big-R Rationalists, who fear the issues of misaligned AI being given power to do bad things.
When they were first talking about this, lots of people ignored this by saying "let's just keep the AI in a box", and even last year it was "what's so hard about an off switch?".
The problem with any model you can just download and run is that some complete idiot will do that and just give the AI agency they shouldn't have. Fortunately, for now the models are more of a threat to their users than anyone else — lawyers who use it to do lawyering without checking the results losing their law licence, etc.
But that doesn't mean open models are not a threat to other people besides their users, as all the artists complaining about losing work due to Stable Diffusion, the law enforcement people concerned about illegal porn, election interference specialists worried about propaganda, and anyone trying to use a search engine, and that research lab that found a huge number of novel nerve agent candidates whose precursors aren't all listed as dual use, will all tell you for different reasons.
Re: Gemma: New Open Models
#193The fact Gemma team is in the comments section answering questions is praiseworthy to me :)
Re: Gemma: New Open Models
#194Tried inference with the 7B model and without flash attention this is soooooo slow. With flash attention the fine-tunning requires A100 or H100.
Also the inference doesn't always stop generating resulting in garbage being added to the response.
Re: Gemma: New Open Models
#195Hopefully not totally gimped like Gemini. Are they releasing an uncensored version?
These are downloadable open models that can be fined tuned. They are the opposite of censored. If you have the motivation, you can bias them however you please.
Is “the opposite of censored” accurate for something that’s default and considerably easier to access mode of operation won’t say many things for sociopolitical reasons? Able to be un censored, sure, but the extent of that is debatable as well.
Re: Gemma: New Open Models
#196Re: Gemma: New Open Models
#197Earlier quoted context omitted.
Honestly, this is more of a PR stunt to advertise the Google Dev ecosystem than a contribution to open-source. I'm not complaining, just calling it what it is. Barely an improvement over the 5-month-old Mistral model, with the same context length of 8k. And this is a release after their announcement of Gemini Pro 1.5, which had an exponential increase in context length.
Who cares if it's a PR stunt to improve developer good will? It's still a good thing, and it's now the most open model out there.
How is it more open than Mistral with Apache 2.0? Google wants people to sign a waiver to even download it.
Re: Gemma: New Open Models
#198Earlier quoted context omitted.
Yes - they are open weights and open inference code, which means they can be integrated into Ollama. They are not “open training” (either in the training code or training data sense), so they are not reproducible, which some have suggested ought to be a component of the definition of open models.
It really should shouldn't it? I'm quite ML-naïve, but surely providing the model without 'training code or training data' is just like providing a self-hostable binary without the source code? Nobody calls that open source, it's not even source available.
It is widely believed (and in some cases acknowledged) that a lot of models are trained on copyrighted data scraped from the web. In some cases, even scrapes of ebook piracy websites - google 'books3' to learn more.
Some companies (such as those working on AI) believe this is legal, others (such as the copyright holders to those books) believe it isn't.
In any case, IMHO it's unlikely any cutting edge models will be offering us their training data any time soon.
Re: Gemma: New Open Models
#199Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-…
the real gold will be when this gets finetuned. (maybe by mistral...)
[deleted]
Re: Gemma: New Open Models
#200Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
I find the snyde remarks around open source in the paper and announcement rather off putting. As the ecosystem evolves, we urge the corporate AI community to move beyond demanding to be taken seriously as a player in open source for models that are not actually open, and avoid preaching with a PR statement that can be interpreted as uniformed at best or malicious at worst.
Working at google is like this, where no matter how much you try to do the right thing you're always under attack.