Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

151–160 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#151

Post by dofm »

minimaxir wrote: post #142
dofm wrote: post #127

Earlier quoted context omitted.

I would contend that the actual big story is the gallery app: https://developers.google.com/edge/gallery Anyone with a 16GB Mac — that is quite a lot of journalists, surely — can download that, install a model into it, and play. Surely journalists have to start asking questions at least about OpenAI's consumer revenue projections now. I am a major, major AI cynic, but I decided to be an informed cynic so I've been pl…

I had discounted Edge Gallery because it didn't support system prompts, but now it does so I will give it another go. I believe the implementation does use MTP since I got an update to Gemma-4-E4B on iOS indicating such, and on macOS it's very speedy. However, on my 18GB RAM MacBook Pro, selecting Gemma-4-12B-it results in this error: > The model "Gemma-4-12B-it' requires more memory (RAM) than is available on your d…

Interesting; they may have fluffed up somewhere then.

(Though perhaps it'll squeeze in with a small context window? Not sure I understand that aspect yet)

It does seem to use MTP, yes, and it is quite quick — seemingly the underlying LiteRT stuff can do MTP with Gemma 4 and presumably MTP is a big part of the practicality picture here.

The system prompt thing was a surprise when I poked around.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#152

Post by verdverm »

mchusma wrote: post #143
anonova wrote: post #135

Do Gemma 4 models compete with Gemini 3.1 Flash-Lite? I would assume even the smallest Gemini model would outperform even Gemma 4 31B, but I can't really get a sense of performance or output quality difference.

Gemma 4 31b outperformed Gemini 3.1 Flash-Lite in our app benchmarks (agentic tool use via api in our application as a part of various workflows). But google won't let you pay to use Gemma models, you have to go elsewhere, I think this may be because it would cannabilize Flash-lite.

You can actually get the gemma-4 models on a per-token API basis, you just have to click some extra buttons (in GCP). Not the same for other open weight models. For those they make you run your own hardware.

Use OpenCode Go instead: https://opencode.ai/go

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#153

Post by literalAardvark »

gdiamos wrote: post #113
literalAardvark wrote: post #74

Earlier quoted context omitted.

I think this plays a part, but the truth is that Google doesn't need to do that, Chinese open models are already doing that by themselves. So perhaps another part is just Google showing that they can indeed play at the big boys table.

There is demand for US open models.

I sincerely wonder why. Chinese censorship is only really relevant if you're doing anti China stuff, which is to say never, while the Western kind of model censorship ( a combination of copyrights and general fairness ) are something everyone's had to work around at least once, even if just for writing an interesting story.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#156

Post by verdverm »

brianwawok wrote: post #34
nickandbro wrote: post #3

Wow Google is becoming the new pre Llama 4 Meta when it comes to releasing open weights models.

Every other Google model I have tried felt very weak compared to qwen models. I dont have a ton of use case for multimodal though, so its very possible this is a fantastic multimodal model.

qwen3.6 was my favorite, then I tried the deepseek-v4-{flash,pro}

still making my way through deep dives on the chinese open weights, they are all pretty good and way more cost / resource effective

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#157

Post by verdverm »

satvikpendem wrote: post #100
lxgr wrote: post #60

Am I missing something or are the Ollama versions of this ( https://ollama.com/library/gemma4/tags ) text-only for now?

Just use llama.cpp or Unsloth Studio which wraps it, I don't know why anyone use Ollama anymore.

I switched from llama.cpp to vLLM because of prompt cache bugs in qwen/gemma models

This is a good starting issue with a bunch of linked/related

https://github.com/ggml-org/llama.cpp/issues/22746

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#158

Post by bityard »

asim wrote: post #137

We are now entering the closed loop game. Google doesn't need anyone else to accelerate their models. This is their bread and butter. I'm both shocked but also not surprised that they continue to develop such efficiencies. Honestly it's like silicon and CPU architecture advancement. We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x…

> We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x more efficient with time.

It's definitely an exciting time, but in terms of advancements in the state of the art, there is a lot of low-hanging fruit left to pick. There IS a bottom, however, as you can only encode so much "knowledge" in a small number of parameters.

This feels to me a lot like what the early days of what radio or aviation must have been like. Or, heck, microcomputers even.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#159

Post by lambda »

matja wrote: post #43
minimaxir wrote: post #2

The big story here is the encoder-free part, which I still don't fully understand. > Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. That's technically encoding, just without using a dedicated model for it like SigLIP? The Developer's Guide elaborates, it's still a 35M layer which I am curious is ro…

One side-effect, is that the separate .mmproj file (Multi-Modal Projection encoder) is no longer needed, when using the model with llama.cpp etc.

It's not? There's an mmproj in the GGUFs released by ggml-org: https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/mai...

From the visual guide, there's still the 35M parameter embedder, then the linear projector, for vision, and the linear projector for audio, so it does have some parameters used for the multimodal input to project it into the LLM latent space: https://newsletter.maartengrootendorst.com/p/a-visual-guide-...

And the Unsloth quants, which are missing this, don't support multimodal input. (edit: actually, I may have just needed to update my llama.cpp, will check with an updated llama.cpp soon)

I'm downloading the ggml-org GGUFs now, I tried Unsloth but got some weird problems, double checking with the bf16 model to see if the issue was just the quant.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#160

Post by santiagobasulto »

spott wrote: post #78
minimaxir wrote: post #2

The big story here is the encoder-free part, which I still don't fully understand. > Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. That's technically encoding, just without using a dedicated model for it like SigLIP? The Developer's Guide elaborates, it's still a 35M layer which I am curious is ro…

This is just early fusion basically. FAIR did this 2 years ago now: https://arxiv.org/abs/2405.09818 I've been waiting for something like this to be released since then. The annoying thing is that chameleon was multi-modal out based on the same principles, but this model is just inputs... (I'm curious how they did pre-training without having multi-modal outputs as well. I wonder if they just chopped them off rather t…

I don't think it's the same. It's a similar concept, but Gemma is using just a linear projection, which I assume is a lot faster. The developer guide has more details: https://developers.googleblog.com/gemma-4-12b-the-developer-...

    Vision embedder (35M parameters): Replaces the 27 vision transformer layers of the other medium-sized Gemma 4 models. Raw 48x48 pixel patches are projected to the LLM hidden dimension with a single matmul. A factorized coordinate lookup (X and Y matrices) attaches spatial location information directly to the input
the "single matmul" is the key here, I haven't tried it, but it's probably pretty fast and memory efficient.
Post reply on HN