Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 2: Improving Open Language Models at a Practical Size [pdf]

storage.googleapis.com

Post reply on HN
Print view View original

171–180 of 183 posts

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#171

Post by sujumayas »

When I used it with ollama in the terminal (first try prompt: "create a snake game in HTML canvas", nothing else) it went forever rambling. It started with the right answer in HTML code, but then it started explaining, and started repeating itself, and then it started to put things like random code snippets and random explanations that were nonsense like:

```python def solve_quadratic_equation(a, b, c): """Solves a quadratic equation of the form ax^2 + bx + c = 0."""

  discriminant = (b ** 2) - (4 * a * a)
  if discriminant >= 0:
    root = (-b + math.sqrt(b ** 2 - 4 * a * a ** b**

  0.5 #
  1.

  # Return None if the quadratic equation has no real roots
  if (b ** 2)
# Print the solution to the equation if (b * 2)

print("The quadratic equation is: " + a * x* 2 + b "x" + c) ```

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#172

Post by sujumayas »

sujumayas wrote: post #171

When I used it with ollama in the terminal (first try prompt: "create a snake game in HTML canvas", nothing else) it went forever rambling. It started with the right answer in HTML code, but then it started explaining, and started repeating itself, and then it started to put things like random code snippets and random explanations that were nonsense like: ```python def solve_quadratic_equation(a, b, c): """Solves a q…

Just realized that Gemma2 is pretty bad in programming tasks. Lol.

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#173

Post by [deleted] »

luke-stanley wrote: post #60
alekandreev wrote: post #6

Hello (again) from the Gemma team! We are quite excited to push this release out and happy to answer any questions! Opinions are our own and not of Google DeepMind.

It's fairly easy to pay OpenAI or Mistral money to use their API's. Figuring out how Google Cloud Vertex works and how it's billed is more complicated. Azure and AWS are similar in how complex they are to use for this. Could Google Cloud please provide an OpenAI compatible API and service? I know it's a different department. But it'd make using your models way easier. It often feels like Google Cloud has no UX or end…

[deleted]

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#174

Post by occamrazor »

lhl wrote: post #169
aubanel wrote: post #73

It's exceptionally strong. In LMSys Chatbot Arena, the 27B version scores above LLama-3-70B, at the level of OpenAI GPT-4 and Claude-3 Sonnet!

Just saw this, might get lost in the noise, but just for posterity, apparently the Gemma 2 models were specifically RL’d to index on Chat Arena performance: https://x.com/natolambert/status/1806384821826109597 (Relevant sections of the paper highlighted.)

On prompts only, with answers presumably from the teacher model (Gemini).

It was not trained or RLHFd on Arena replies or user preferences.

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#175

Post by groby_b »

astrange wrote: post #152
groby_b wrote: post #121

Earlier quoted context omitted.

You're confusing it with data poisoning. Model collapse itself is(was?) a fairly serious research topic: https://arxiv.org/abs/2305.17493 We've by now reached a "probably not inevitable" - https://arxiv.org/abs/2404.01413 argues there's a finite upper bound to error - but I'd also point out that that paper assumes training data cardinality increases with the number of training generations and is strictly accumulative…

Oh, no, they definitely believe both are going to happen and ChatGPT is just going to stop working because it'll see itself on the internet. It goes with the common belief that LLMs learn from what you type into them. > To a first order, that means you better have a pre-2022 dataset to get started, and have archived it well. I think that will always be available, or at least, a dataset with the distribution you want…

Don't know why you have such a disdain for artists, but either way, the original point was that model collapse wasn't "a coping idea made up by artists", but a valid research backed scientific model.

>I think that [clean pre-2022 data set] will always be available

Good luck obtaining one.

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#176

Post by alecco »

moffkalast wrote: post #161
magicalhippo wrote: post #141

Earlier quoted context omitted.

Ironically I was just thinking earlier today how the most valuable Google products to me are YouTube and Android... and that's it. I gave up on Chrome a decade ago, going back to Firefox. I don't use Google for search anymore, I do use Gmail but I also got Protonmail so could easily migrate the Gmail traffic there. A lot of non-techies I know have complained for some time how Google search sucks, and while a lot use…

Is it really possible to even disrupt Youtube? It's been a constant in our lives for the past 20 years and is basically a historical record by now. By a rough estimate, they have to keep buying over 1% of the total world production of HDD drives just to stay on top of the new data being uploaded. Google has completely destroyed it, placing more ads than videos on it, making it unusable without an adblocker and people…

I've been thinking about it. AI generated videos. It could be generating a DSL or IR for some sort of multimedia VM so there's only a tiny fraction of data. Just common textures and shapes in a CDN. Could be fully interactive.

I wouldn't be surprised if most of it was already tried in some form.

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#177

Post by luke-stanley »

janwas wrote: post #153
thot_experiment wrote: post #142

Earlier quoted context omitted.

I'm curious about the quantization quality claims in the table there. Is this a Gemma 2 specific thing (more subtlety in the weights somehow?). In my testing and testing I've seen elsewhere at least for llama3 8B (and some less rigorous testing with other models) q_8 -> q4_K_M are basically indistinguishable from one another?

Yes, PPL and certain benchmarks do not detect differences from quantization. But recent work gives cause for concern, e.g., https://arxiv.org/pdf/2310.01382 , https://arxiv.org/pdf/2405.18137 .

The first paper is good to critique the performance of quantised models, it points out that 40-50% 'compression' typically results in only slight loss for RAG tasks relying on in-context learning, but for factual tasks replying on stored knowledge, performance very quickly dropped off. They looked at Vicuna, one of the earlier models, so I wonder how applicable it is to recent models like the Phi 3 range. I don't think deliberate clever adversarial attacks like those of the 2nd paper are a sensible worry for most, but it is fun. Thanks for the links @janwas.

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#178

Post by imjonse »

viridian wrote: post #167
imjonse wrote: post #156

Earlier quoted context omitted.

It will refuse to describe the process of making napalm using only double entendres.

I don't understand the point of this sort of censorship when I can go to google, ask how to make napalm, and get a million results telling me to dissolve styrofoam in gasoline. I've seen documentaries and science shows on cable TV that demonstrate basic facts like this, or how the IRA produced IEDs, or how molotov cocktails were made in the spanish civil war. The information is beyond easy to access, and has been for…

True. But an LLM product is closely associated with a single company and unlike a search engine which can claim it only shows you what is already available, the LLM will seem like it personally tells you something harmful. When they want to sell it as a helpful assistant that kind of behavior will undermine that goal.

We saw all the bad press companies have got in recent years for all kinds of unintended AI outputs.

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#179

Post by Grimblewald »

smcleod wrote: post #113

It has a tiny context window of 8k, that thing will have the memory of a goldfish.

8K is a sizable window, sure larger 'exists' but also advertised context windows and functional context windows are not the same thing. I would rather a model that can 'only' handle 8k tokens but handles 8k as well as it handles 1k compared to a model that 'can' handle 32k, but realistically, output for contexts beyond 1k are garbage.

Re: Gemma 2: Improving Open Language Models at a Practical Size [pdf]

#180

Post by smcleod »

Grimblewald wrote: post #179
smcleod wrote: post #113

It has a tiny context window of 8k, that thing will have the memory of a goldfish.

8K is a sizable window, sure larger 'exists' but also advertised context windows and functional context windows are not the same thing. I would rather a model that can 'only' handle 8k tokens but handles 8k as well as it handles 1k compared to a model that 'can' handle 32k, but realistically, output for contexts beyond 1k are garbage.

Deepseek Coder v2 and Qwen2 are both great at 32k context. Can’t tell the difference between those models at 8k and 32k fully utilised. The difference in quality between them and 8k models when doing codegen is night and day. Not to mention that many of the little 8k models also have sliding window at 4k which essentially makes them 4k models.
Post reply on HN