351–360 of 543 posts
If you are looking for a nice chat UI to try out Gemma (and other offline + online models) locally, I'm working on an app [1] that is offline and privacy focused. I've just added support for Gemma 7B. [1]: https://msty.app
One usage question: after you've downloaded a model and are finished trying it out, how do you remove it?
I personally can't take any models from google seriously. I was asking it about the Japanese Heian period and it told me such nonsensical information you would have thought it was a joke or parody. Some highlights were "Native American women warriors rode across the grassy plains of Japan, carrying Yumi" and "A diverse group of warriors, including a woman of European descent wielding a katana, stand together in camar…
I was wondering if these models would perform in such a way, given this week's X/twitter storm over Gemini generated images. E.g. https://x.com/debarghya_das/status/1759786243519615169?s=20 https://x.com/MiceynComplex/status/1759833997688107301?s=20 https://x.com/AravSrinivas/status/1759826471655452984?s=20
Why are there three links to this question? And why are people so upset over it? Very odd, seems like it is mostly driven by political rage.
Earlier quoted context omitted.
Strong disagree - a Mistral fine tune of llama 70b was the top performing llama fine tune. They have lots of data the community simply does not.
Miqu was (allegedly) an internal continued pretrain Mistral did as a test, that was leaked as a GGUF. Maybe its just semantics, it is technically a finetune... But to me theres a big difference between expensive "continuation training" (like Solar 10.7B or Mistral 70B) and a much less intense finetuning. The former is almost like releasing a whole new base model. It would be awesome if Mistral did that with their dat…
The terms of use: https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policy Something that caught my eye in the terms: > Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma. One of the biggest benefits of running your own model is that it can protect you from model updates that break your carefully tested prompts, so I’m not…
They have to make sure you’re receiving the most cutting edge chiding lectures when you make naughty and problematic requests.
Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
I work on Ollama and used the provided GGUF files to quantize the model. As mentioned by a few people here, the 4-bit integer quantized models (which Ollama defaults to) seem to have strange output with non-existent words and funny use of whitespace.
Do you have a link /reference as to how the models were converted to GGUF format? And is it expected that quantizing the models might cause this issue?
Thanks so much!
Earlier quoted context omitted.
I also saw someone prompt it for "German couple in the 1800s" and, while I'm not trying to paint Germany as ethnically homogenous, 3 out of the 4 images only included Black, Asian or Indigenous people. Which, especially for the 19th century with very few travel options, seems like a super weird choice. They are definitely heavily altering prompts.
There's one in the comments of yesterday's Paul Graham Twitter thread where someone prompted Gemini with "Generate an image of German soldiers in 1943" and it came back with a picture of a black guy and an Asian woman in Nazi uniforms on the battlefield. If you specifically prompt it to generate an image of white German soldiers in 1943 it will tell you it can't do that because it's important that we maintain diversi…
I notice a few divergences to common models: - The feedforward hidden size is 16x the d_model, unlike most models which are typically 4x; - The vocabulary size is 10x (256K vs. Mistral’s 32K); - The training token count is tripled (6T vs. Llama2's 2T) Apart from that, it uses the classic transformer variations: MQA, RoPE, RMSNorm. How big was the batch size that it could be trained so fast? https://huggingface.co/mis…
Damn, 6T? That's a lot!
Given that this model seems to roughly match Mistral (according to the numbers from Google), this makes me think we have saturated the 7B parameter space, and couldn't possibly make it much better unless new techniques are discovered.
The terms of use: https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policy Something that caught my eye in the terms: > Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma. One of the biggest benefits of running your own model is that it can protect you from model updates that break your carefully tested prompts, so I’m not…
sounds good.
this is not financial advice and ianal.
Earlier quoted context omitted.
Huh, indeed, that's what the config.json[0] says; the report[1] indicates “Feedforward hidden dims: 49152”. [0]: https://huggingface.co/google/gemma-7b-it/blob/main/config.j... [1]: https://storage.googleapis.com/deepmind-media/gemma/gemma-re...
I don't see the number 49152 reported in the config.json, what line are you referring to? I just see the intermediate_size of 24576 (so 8x). EDIT: I didn't read the comment correctly, you have noticed the same thing.
In the report, "hidden dim" matches the number of GEGLU inputs. In the config, "intermediate_size" matches the number of GEGLU outputs. Most *GLU models so far have used intermediate_size=8/3*d_model as this makes have the same number of matmul FLOPS & parameters as a 4x-expanded non-GLU model, and PaLM vaguely showed that 4x is better than a smaller expansion factor.
If one considers Llama-2-7B's FFN expansion factor to be ~5.33x, Gemma's expansion factor is 16x.
I notice a few divergences to common models: - The feedforward hidden size is 16x the d_model, unlike most models which are typically 4x; - The vocabulary size is 10x (256K vs. Mistral’s 32K); - The training token count is tripled (6T vs. Llama2's 2T) Apart from that, it uses the classic transformer variations: MQA, RoPE, RMSNorm. How big was the batch size that it could be trained so fast? https://huggingface.co/mis…
What does tokenization look like in 256k vs 32k?