291–300 of 507 posts
For token-generation speed, a challenging test is to see how it performs in a code-agent harness like Claude Code, which has anywhere between 15-40K tokens from the system prompt itself (+ tools/skills etc). Here the 26B-A4B variant is head and shoulders above recent open-weight models, at least on my trusty M1 Max 64GB MacBook. I set up Claude Code to use this variant via llama-server, with 37K tokens initial contex…
I ran these in LM Studio and got unrecognizable pelicans out of the 2B and 4B models and an outstanding pelican out of the 26b-a4b model - I think the best I've seen from a model that runs on my laptop. https://simonwillison.net/2026/Apr/2/gemma-4/ The gemma-4-31b model is completely broken for me - it just spits out "---\n" no matter what prompt I feed it. I got a pelican out of it via the AI Studio API hosted model…
Do you have a single gallery page where we can see all the pelicans together. I'm thinking something similar to https://clocks.brianmoore.com/ but static.
Earlier quoted context omitted.
Following the current rule of thumb MoE = `sqrt(param*active)` a 200B-A3B would have the intelligence of a ~24B dense model. That seems pointless. You can achieve that with a single 24G graphics card already. I wonder if it would even hold up at that level, as 3B active is really not a lot to work with. Qwen 3.5 uses 122B-A10B and still is neck and neck with the 27B dense model. I don't see any value proposition for…
That rule of thumb was invented years ago, and I don’t think it is relevant anymore, despite how frequently it is quoted on Reddit. It is certainly not the "current" rule of thumb. For the sake of argument, even if we take that old rule of thumb at face value, you can see how the MoE still wins: - (DGX Spark) 273GB/s of memory bandwidth with 3B active parameters at Q4 = 273 / 1.5 = 182 tokens per second as the theore…
I ran these in LM Studio and got unrecognizable pelicans out of the 2B and 4B models and an outstanding pelican out of the 26b-a4b model - I think the best I've seen from a model that runs on my laptop. https://simonwillison.net/2026/Apr/2/gemma-4/ The gemma-4-31b model is completely broken for me - it just spits out "---\n" no matter what prompt I feed it. I got a pelican out of it via the AI Studio API hosted model…
Mind I ask what your laptop is and configuration hardware wise?
Prompt: > what is the Unix timestamp for this: 2026-04-01T16:00:00Z Qwen 3.5-27b-dwq > Thought for 8 minutes 34 seconds. 7074 tokens. > The Unix timestamp for 2026-04-01T16:00:00Z is: > 1775059200 (my comment: Wednesday, 1 April 2026 at 16:00:00) Gemma-4-26b-a4b > Thought for 33.81 seconds. 694 tokens. > The Unix timestamp for 2026-04-01T16:00:00Z is: > 1775060800 (my comment: Wednesday, 1 April 2026 at 16:26:40) Gem…
For token-generation speed, a challenging test is to see how it performs in a code-agent harness like Claude Code, which has anywhere between 15-40K tokens from the system prompt itself (+ tools/skills etc). Here the 26B-A4B variant is head and shoulders above recent open-weight models, at least on my trusty M1 Max 64GB MacBook. I set up Claude Code to use this variant via llama-server, with 37K tokens initial contex…
gpt oss 20b is not dense
Earlier quoted context omitted.
What was the main focus when training this model? Besides the ELO score, it's looking like the models (31B / 26B-A4) are underperforming on some of the typical benchmarks by a wide margin. Do you believe there's an issue with the tests or the results are misleading (such as comparative models benchmaxxing)? Thank you for the release.
Becnhmarks are a pox on LLMs. You can use this model for about 5 seconds and realize its reasoning is in a league well above any Qwen model, but instead people assume benchmarks that are openly getting used for training are still relevant.
So what's the business strategy here? Google is the only USA based frontier lab releasing open models. I know they aren't doing it out of the goodness of their hearts.
Release open weights so competitors can't raise good money, then rear naked choke when they run dry
Earlier quoted context omitted.
How do the smaller models differ from what you guys will ultimately ship on Pixel phones? What's the business case for releasing Gemma and not just focusing on Gemini + cloud only?
Its hard to say because Pixel comes prepacked with a lot of models, not just ones that that are text output models. With the caveat that I'm not on the pixel team and I'm not building _all_ the models that are on google's devices, its evident there are many models that support the Android experience. For example the one mentioned here https://store.google.com/us/magazine/magic-editor?hl=en-US&p...
Hi all! I work on the Gemma team, one of many as this one was a bigger effort given it was a mainline release. Happy to answer whatever questions I can
https://aibenchy.com/compare/google-gemma-4-31b-it-medium/go...