Gemma: New Open Models • Hacker News Board

Re: Gemma: New Open Models

#541

Post by trisfromgoogle » Fri, Feb 23, 2024, 8:12 PM UTC

artninja1988 wrote: post #155

trisfromgoogle wrote: post #131

Earlier quoted context omitted.

Quick question -- can you tell me where you got that quote? It's not in the main blog or any of the launch communications that I can see.

The quote is from the technical report https://storage.googleapis.com/deepmind-media/gemma/gemma-re...

Ah, thanks for clarifying! It's a good flag, though I wouldn't classify it as a snide comment personally. I'd be interested in hearing what you find snide or offensive about it -- do you think we shouldn't be trying to bring the whole community along for evals/safety/etc, regardless of open/closed?

Re: Gemma: New Open Models

#542

Post by kleiba » Sat, Feb 24, 2024, 1:56 PM UTC

alekandreev wrote: post #463

kleiba wrote: post #383

Earlier quoted context omitted.

> We are really excited to answer any questions you may have about our models. I cannot count how many times I've seen similar posts on HN, followed by tens of questions from other users, three of which actually get answered by the OP. This one seems to be no exception so far.

Sorry, doing our best here :)

Thank you!

Re: Gemma: New Open Models

#543

Post by lobocinza » Sun, Feb 25, 2024, 2:19 AM UTC

kgeist wrote: post #532

kgeist wrote: post #414

Earlier quoted context omitted.

What prompts/settings do you use for Phi-2? I found it completely unusable for my cases. It fails to follow basic instructions (I tried several instruction-following finetunes as well, in addition to the base model), and it's been mostly like a random garbage generator for me. With Llama.cpp, constrained to JSON, it also often hangs because it fails to find continuations which satisfy the JSON grammar. I'm building a…

An update on my endeavour: so, model switching is very costly under llama.cpp (I have to switch between Llama and Phi2 because my GPU has low amounts of VRAM). And this switch (reloading the weights into VRAM) defeats the whole purpose of the optimization. Having only Llama on GPU without reloading takes less time than if I'd use Llama+Phi2. And Phi2 alone is pretty bad as a general purpose LLM. So I'm quite disappoi…

I recently upgraded to AM5 and as I have an AMD GPU I'm using llama.cpp on CPU only and I was positively surprised by how fast it generate stuff. I don't have the case of massive workloads so YMMV.