Gemma 4 12B: A unified, encoder-free multimodal model • Hacker News Board

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#131

Post by SeriousM » Wed, Jun 03, 2026, 6:35 PM UTC

properbrew wrote: post #80

djyde wrote: post #36

What are the use cases for these small models? Is there anyone using models of this scale in their daily life who could share their experience?

I think small models have a very good niche for specific tasks. I utilise a fine tuned Phi-4 model (smaller than this one) that fits in about 3.5gb of RAM (not vram) for the document processing side of things for the desktop app I develop (a bit of a shameless plug - whistle-enterprise.com). If you have a very specific idea for local model use you can find a way to make it work very well, you don't even need to have…

Thank you for sharing your usecase! I like your product very much!

Could you talk a bit how you did the finetuning? Did you use unsloth or any other tool and how went the verification to proof the outcome?

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#132

Post by petercooper » Wed, Jun 03, 2026, 6:35 PM UTC

Its image processing is terrible. I ran several tests against it against Qwen 3.5 0.8b (yes, 7% the size) and Qwen beat it every time with Gemma often getting things entirely wrong. I even gave it a plain image saying "This is a test" and it thought for 6 minutes trying to analyze it and failed. Qwen 3.5 0.8b confidently got it in under a second.

It may be that the Q6 quant I got is borked (or my LM Studio is), but either way, the 0.8b's performance is mind boggling in comparison.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#133

Post by semiinfinitely » Wed, Jun 03, 2026, 6:35 PM UTC

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#134

Post by claysmithr » Wed, Jun 03, 2026, 6:36 PM UTC

corgihamlet wrote: post #114

claysmithr wrote: post #56

I don’t see the download in lm studio

https://lmstudio.ai/models/gemma-4

Thanks looks like they just added it 1 hour ago

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#135

Post by anonova » Wed, Jun 03, 2026, 6:39 PM UTC

Do Gemma 4 models compete with Gemini 3.1 Flash-Lite? I would assume even the smallest Gemini model would outperform even Gemma 4 31B, but I can't really get a sense of performance or output quality difference.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#136

Post by [deleted] » Wed, Jun 03, 2026, 6:45 PM UTC

ethanpil wrote: post #13

What's Google's business case for releasing open models? Don't get me wrong, I am grateful and appreciative of these releases. I'm trying to understand how it fits into their bigger picture as a for profit company? Are they not helping competitors build on the novel technology they have developed? Is it simply goodwill and/or marketing? Or am I missing something strategic?

[deleted]

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#137

Post by asim » Wed, Jun 03, 2026, 6:45 PM UTC

We are now entering the closed loop game. Google doesn't need anyone else to accelerate their models. This is their bread and butter.

I'm both shocked but also not surprised that they continue to develop such efficiencies. Honestly it's like silicon and CPU architecture advancement. We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x more efficient with time. Maybe there's some point of decay but essentially the next 30 years will be more advanced than the last 30 and were going to be living in some sort of futurist blade runner scenario where gene editing is repairing ageing cells, organs and curing all sorts of cancers that haven't even appeared yet. Beyond our lifetimes people will live to 125 quite steadily and with great mobility and then obviously people will look to how do we get to living 1000 years, which of anyone is religious knows Noah and others lived to that age in a totally different era.

Anyway I'm going off on some tangent but look back 30 years. Now look forward 30 years. It's going to be insane. May God protect us.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#138

Post by [deleted] » Wed, Jun 03, 2026, 6:49 PM UTC

digdugdirk wrote: post #50

I do enjoy the immediate out of touch signaling with the "runs on your 16gb vram laptop" line. Because everyone has a laptop with 16gb vram, or can just pop out and buy a new one, right?

[deleted]

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#139

Post by mchusma » Wed, Jun 03, 2026, 6:52 PM UTC

ethanpil wrote: post #13

What's Google's business case for releasing open models? Don't get me wrong, I am grateful and appreciative of these releases. I'm trying to understand how it fits into their bigger picture as a for profit company? Are they not helping competitors build on the novel technology they have developed? Is it simply goodwill and/or marketing? Or am I missing something strategic?

I think its even more puzzling because you can't even run Gemma 31b on google cloud, they only let you test it with a rate limit. No way (I can find) to actually pay them to use it.

We saw great results in our usecase using google direct. Moved to Openrouter because google wouldn't let us use it beyond a test.

Then Openrouters performance looked worse, not sure if there was a quantized version or something. So we instead looked at Deepseek v4 Flash, and opted to go for that.

This model would probably be great for a super low cost cloud model, would love to use it in the cloud, Google makes you go elsewhere.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#140

Post by madduci » Wed, Jun 03, 2026, 6:53 PM UTC

minimaxir wrote: post #2

The big story here is the encoder-free part, which I still don't fully understand. > Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. That's technically encoding, just without using a dedicated model for it like SigLIP? The Developer's Guide elaborates, it's still a 35M layer which I am curious is ro…

VRAM, not RAM. I wish it was light enough for iGPUs too