31–40 of 183 posts
So it's twice the size of phi 3 and considerably worse? What am I missing
I imagine Gemma 2 is a better general-purpose assistant for most people, whereas Phi 3 is a solid small LLM (SLM?) for more specific use-cases like summarization, RAG, learning about math and stuff.
Phi-3 blow this out of the water. Benchmark | Gemma 2 (9B) | Phi-3 Small (7B) -----------------------------|----------------|------------------- MMLU (5-Shot) | 63.6 | 75.7 HellaSwag (5-Shot) | 49.8 | 77.0 ANLI (7-Shot) | 48.7 | 58.1 GSM-8K (8-Shot; CoT) | 59.8 | 89.6 MedQA (2-Shot) | 49.6 | 65.4 AGIEval (0-Shot) | 42.1 | 45.1 TriviaQA (5-Shot) | 72.3 | 58.1 Arc-C (10-Shot) | 78.3 | 90.7 Arc-E (10-Shot) | 91.4 | 97.0…
Hello (again) from the Gemma team! We are quite excited to push this release out and happy to answer any questions! Opinions are our own and not of Google DeepMind.
Are the weights released yet?
Are these small Gemma 2 distilled models available anywhere? I'm not finding them on huggingface.co, etc. but maybe I don't know the exact model names they are published. Are the weights released yet?
Are these small Gemma 2 distilled models available anywhere? I'm not finding them on huggingface.co, etc. but maybe I don't know the exact model names they are published. Are the weights released yet?
> Table 4 | Relevant formatting control tokens used for Gemma models > User turn: user > Model turn: model > Start of conversation turn: > End of conversation turn: > Beginning of sequence: > End of sequence: You know I keep wondering why and tokens are even a thing in general. No model is tuned to keep generating multiple turns after its equivalent is sent, and what's the point of when you're parsing the entire cont…
Your training input has the shape of (sequence length x batch size). If a lot of your samples are shorter than sequence length, as is usually the case, you will have a lot of padding tokens in the input, which is wasted compute. To compensate for that, you can pack multiple examples in the same sequence. This is there EOS and BOS come in, as they indicate to the model that the two parts of the sequence are not relate…
Hello (again) from the Gemma team! We are quite excited to push this release out and happy to answer any questions! Opinions are our own and not of Google DeepMind.
Any gemma-2-9b or 27b 4 bit GGUF's on HuggingFace yet? Thanks!
So it's twice the size of phi 3 and considerably worse? What am I missing