Fyi, it took me a while to find the meaning of the "-it" in some models. That's how Google designates "instruction tuned". Come on Google. Definite your acronyms.
Google releases Gemma 4 open models
161–170 of 507 posts
Re: Google releases Gemma 4 open models
#162Earlier quoted context omitted.
Instruction Tuned. It indicates that thinking tokens (eg ) are not included in training.
That’s not what it means. "-it" just indicates the model is instruction-tuned, i.e. trained to follow prompts and behave like an assistant. It doesn’t imply anything about whether thinking tokens like .... were included or excluded during training. Thats a separate design choice and varies by model.
What does that mean for a user of the model? Is the "-it" version more direct with solutions or something?
Re: Google releases Gemma 4 open models
#163Gemma 3 models were pretty bad, so hopefully they got Gemma 4 to at least come close to the other major open weights
Re: Google releases Gemma 4 open models
#164So the "E2B" and "E4B" models are actually 5B and 8B parameters. Are we really going to start referring to the "effective" parameter count of dense models by not including the embeddings? These models are impressive but this is incredibly misleading. You need to load the embeddings in memory along with the rest of the model so it makes no sense o exclude them from the parameter count. This is why it actually takes 5G…
These are based on the Gemma 3n architecture so E2B only needs 2Gb for text2text generation:
https://ai.google.dev/gemma/docs/gemma-3n#parameters
You can think of the per layer-embeddings as a vector database so you can in theory serve it directly from disk.
Re: Google releases Gemma 4 open models
#165Earlier quoted context omitted.
We are always figuring out what parameter size makes sense. The decision is always a mix between how good we can make the models from a technical aspect, with how good they need to be to make all of you super excited to use them. And its a bit of a challenge what is an ever changing ecosystem. I'm personally curious is there a certain parameter size you're looking for?
120B would be great to have if you have it stashed away somewhere. GPT-OSS-120B still stands as one of the best (and fastest) open-weights models out there. A direct competitor in the same size range would be awesome. The closest recent release was Qwen3.5-122B-A10B.
Nemotron 3 Super was released recently. That's a direct competitor to gpt-oss-120b. https://developer.nvidia.com/blog/introducing-nemotron-3-sup...
Re: Google releases Gemma 4 open models
#166I ran these in LM Studio and got unrecognizable pelicans out of the 2B and 4B models and an outstanding pelican out of the 26b-a4b model - I think the best I've seen from a model that runs on my laptop. https://simonwillison.net/2026/Apr/2/gemma-4/ The gemma-4-31b model is completely broken for me - it just spits out "---\n" no matter what prompt I feed it. I got a pelican out of it via the AI Studio API hosted model…
I'd recommend using the instruction tuned variants, the pelicans would probably look a lot better.
Re: Google releases Gemma 4 open models
#167Earlier quoted context omitted.
downloading the official ones for my m3 max 128GB via lm studio I can't seem to get them to load. they fail for some unknown reason. have to dig into the logs. any luck for you?
The Unsloth llama.cpp guide[1] recommends building the latest llama.cpp from source, so it's possible we need to wait for LM Studio to ship an update to its bundled llama.cpp. Fairly common with new models. 1. https://unsloth.ai/docs/models/gemma-4#llama.cpp-guide
LM Studio shipped this update. Under settings make sure you update your runtimes.
Re: Google releases Gemma 4 open models
#168I ran these in LM Studio and got unrecognizable pelicans out of the 2B and 4B models and an outstanding pelican out of the 26b-a4b model - I think the best I've seen from a model that runs on my laptop. https://simonwillison.net/2026/Apr/2/gemma-4/ The gemma-4-31b model is completely broken for me - it just spits out "---\n" no matter what prompt I feed it. I got a pelican out of it via the AI Studio API hosted model…
Mind I ask what your laptop is and configuration hardware wise?
Re: Google releases Gemma 4 open models
#169Comparison of Gemma 4 vs. Qwen 3.5 benchmarks, consolidated from their respective Hugging Face model cards: | Model | MMLUP | GPQA | LCB | ELO | TAU2 | MMMLU | HLE-n | HLE-t | |----------------|-------|-------|-------|------|-------|-------|-------|-------| | G4 31B | 85.2% | 84.3% | 80.0% | 2150 | 76.9% | 88.4% | 19.5% | 26.5% | | G4 26B A4B | 82.6% | 82.3% | 77.1% | 1718 | 68.2% | 86.3% | 8.7% | 17.2% | | G4 E4B |…
Wild differences in ELO compared to tfa's graph: https://storage.googleapis.com/gdm-deepmind-com-prod-public/... (Comparing Q3.5-27B to G4 26B A4B and G4 31B specifically) I'd assume Q3.5-35B-A3B would performe worse than the Q3.5 deep 27B model, but the cards you pasted above, somehow show that for ELO and TAU2 it's the other way around... Very impressed by unsloth's team releasing the GGUF so quickly, if that's lik…
You're conflating lmarena ELO scores.
Qwen actually has a higher ELO there. The top Pareto frontier open models are:
model |elo |price
qwen3.5-397b-a17b |1449 |$1.85
glm-4.7 |1443 | 1.41
deepseek-v3.2-exp-thinking |1425 | 0.38
deepseek-v3.2 |1424 | 0.35
mimo-v2-flash (non-thinking) |1393 | 0.24
gemma-3-27b-it |1365 | 0.14
gemma-3-12b-it |1341 | 0.11
gpt-oss-20b |1318 | 0.09
gemma-3n-e4b-it |1318 | 0.03
What Gemma seems to have done is dominate the extreme cheap end of the market. Which IMO is probably the most important and overlooked segment
Re: Google releases Gemma 4 open models
#170Earlier quoted context omitted.
What was the main focus when training this model? Besides the ELO score, it's looking like the models (31B / 26B-A4) are underperforming on some of the typical benchmarks by a wide margin. Do you believe there's an issue with the tests or the results are misleading (such as comparative models benchmaxxing)? Thank you for the release.
Becnhmarks are a pox on LLMs. You can use this model for about 5 seconds and realize its reasoning is in a league well above any Qwen model, but instead people assume benchmarks that are openly getting used for training are still relevant.
Definitely have to use each model for your use case personally, many models can train to perform better on these tests but that might not transfer to your use case.