Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 2: Improving Open Language Models at a Practical Size [pdf]

storage.googleapis.com

Post reply on HN
Print view View original

101–110 of 183 posts

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#101

Post by [deleted] »

bapcon wrote: post #69
luke-stanley wrote: post #60

Earlier quoted context omitted.

It's fairly easy to pay OpenAI or Mistral money to use their API's. Figuring out how Google Cloud Vertex works and how it's billed is more complicated. Azure and AWS are similar in how complex they are to use for this. Could Google Cloud please provide an OpenAI compatible API and service? I know it's a different department. But it'd make using your models way easier. It often feels like Google Cloud has no UX or end…

I have to agree with all of this. I tried switching to Gemini, but the lack of clear billing/quotas, horrible documentation, and even poor implementation of status codes on failed requests have led me to stick with OpenAI. I don't know who writes Google's documentation or does the copyediting for their console, but it is hard to adapt. I have spent hours troubleshooting, only to find out it's because the documentatio…

[deleted]

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#102

Post by jpcapdevila »

janwas wrote: post #83
austinvhuang wrote: post #55

Earlier quoted context omitted.

This is in the works in the dev branch (thanks pchx :) https://github.com/google/gemma.cpp/pull/274

:) Confirmed working. We've just pushed the dev branch to main.

Awesome, I love this .cpp trend! Thanks for your work!!

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#103

Post by np_space »

alekandreev wrote: post #6

Hello (again) from the Gemma team! We are quite excited to push this release out and happy to answer any questions! Opinions are our own and not of Google DeepMind.

Are Gemma-2 models available via API yet? Looks to me like it's not yet on vertexai

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#104

Post by staticman2 »

johnfn wrote: post #11
behnamoh wrote: post #7

I gave up hope on r"Gem[ma|ini]" long time ago. I don't believe that Google can't produce good LLMs because of its massive company size; Microsoft is also a giant company (more market cap than Google) but it keeps surprising us with the ϕ models. I think Google just lacks the vision to understand what makes a good LLM. Theoretical contributions by research teams are valuable, but the real-world is built around engine…

Maybe you gave up before Google released Gemini Advanced? This viewpoint seemed more accurate before it was related, but Gemini Advanced is the third best LLM as rated here [1]. In fact, had second place until a few days ago when Claude 3.5 came out. [1]: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...

Isn't Gemini Advanced Gemini Pro attached to some sort of an internet search program? If it has that advantage over other models it isn't a sign of AI chops.

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#105

Post by jug »

moffkalast wrote: post #92
screye wrote: post #86

Earlier quoted context omitted.

What's the most obvious standouts? In my experience, smaller models tend to do well on benchmarks and fail at generalization. Phi-2 comes to mind.

It's multilingual. Genuinely. Compared my results with some people on reddit and the consensus is that the 27B is near perfect in a few obscure languages and likely perfect in most common ones. The 9B is not as good but it's still coherent enough to use in a pinch. It's literally the first omni-translation tool that actually works that you can run offline at home. I'm amazed that Google mentioned absolutely nothing a…

Wow, that's very impressive and indeed a game changer. I've previously had trouble with various Scandinavian languages, but last I checked with was Llama 2 and I kind of gave up on it. I had expected we were going to need special purpose small models for these uses as a crutch, like SW-GPT3.

So I guess Gemma 2 is going to become Gemini 2.0 in their truly large and closed variants then? Or is it the open version of Gemini 1.5?

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#106

Post by luke-stanley »

ankeshanand wrote: post #78
luke-stanley wrote: post #60

Earlier quoted context omitted.

It's fairly easy to pay OpenAI or Mistral money to use their API's. Figuring out how Google Cloud Vertex works and how it's billed is more complicated. Azure and AWS are similar in how complex they are to use for this. Could Google Cloud please provide an OpenAI compatible API and service? I know it's a different department. But it'd make using your models way easier. It often feels like Google Cloud has no UX or end…

If you're an individual developer and not an enterprise, just go straight to Google AIStudio or GeminiAPI instead: https://aistudio.google.com/app/apikey . It's dead simple getting an API key and calling with a rest client.

Interesting but when I tried it, I couldn't figure out the billing model because it's all connected to Google projects, and there can be different billing things for each of them.

Each thing seems to have a bunch of clicks to setup that startup LLM providers don't hassle people with. They're more likely to just let you sign in with some generic third party oAuth, slap on Stripe billing, let you generate keys, show you some usage stats, getting started docs, with example queries and a prompt playground etc.

What about the Vertex models though? Are they all actually available via Google AI Studio?

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#107

Post by luke-stanley »

Deathmax wrote: post #79
luke-stanley wrote: post #60

Earlier quoted context omitted.

It's fairly easy to pay OpenAI or Mistral money to use their API's. Figuring out how Google Cloud Vertex works and how it's billed is more complicated. Azure and AWS are similar in how complex they are to use for this. Could Google Cloud please provide an OpenAI compatible API and service? I know it's a different department. But it'd make using your models way easier. It often feels like Google Cloud has no UX or end…

Gemini models on Vertex AI can be called via a preview OpenAI-compatible endpoint [1], but shoving it into existing tooling where you don't have programmatic control over the API key and is long lived is non-trivial because GCP uses short lived access tokens (and long-lived ones are not great security-wise). Billing for the Gemini models (on Vertex AI, the Generative Language AI variant still charges by tokens) I wou…

Good to know about this API preview. Hopefully the billing problem and UI maze of Vertex AI can be sorted too?

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#108

Post by jerrygenser »

alekandreev wrote: post #62
jerrygenser wrote: post #34

Are these small Gemma 2 distilled models available anywhere? I'm not finding them on huggingface.co, etc. but maybe I don't know the exact model names they are published. Are the weights released yet?

In addition to the HF links shared by sibling comments, the 2B will be released soon.

that's actually the particular one I was looking for and couldn't find. Also had googled for the other ones but maybe it was so recent that it hadn't been indexed. Thanks!

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#109

Post by dinosaurdynasty »

moffkalast wrote: post #97
alekandreev wrote: post #81

Earlier quoted context omitted.

To quote Ludovic Peran, our amazing safety lead: Literature has identified self-proliferation as dangerous capability of models, and details about how to define it and example of form it can take have been openly discussed by GDM ( https://arxiv.org/pdf/2403.13793 ). Current Gemma 2 models' success rate to end-to-end challenges is null (0 out 10), so the capabilities to perform such tasks are currently limited.

Turns out LLM alignment is super easy, barely an inconvenience.

One should not confuse alignment and current incapability.

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#110

Post by josh-sematic »

moffkalast wrote: post #97
alekandreev wrote: post #81

Earlier quoted context omitted.

To quote Ludovic Peran, our amazing safety lead: Literature has identified self-proliferation as dangerous capability of models, and details about how to define it and example of form it can take have been openly discussed by GDM ( https://arxiv.org/pdf/2403.13793 ). Current Gemma 2 models' success rate to end-to-end challenges is null (0 out 10), so the capabilities to perform such tasks are currently limited.

Turns out LLM alignment is super easy, barely an inconvenience.

Alignment is tight!
Post reply on HN