Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

171–180 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#171

Post by thot_experiment »

petercooper wrote: post #132

Its image processing is terrible. I ran several tests against it against Qwen 3.5 0.8b (yes, 7% the size) and Qwen beat it every time with Gemma often getting things entirely wrong. I even gave it a plain image saying "This is a test" and it thought for 6 minutes trying to analyze it and failed. Qwen 3.5 0.8b confidently got it in under a second . It may be that the Q6 quant I got is borked (or my LM Studio is), but…

I've always found the Gemma models to vastly under-perform on vision tasks compared to Qwen so that's nothing new.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#172

Post by OtherShrezzing »

djyde wrote: post #36

What are the use cases for these small models? Is there anyone using models of this scale in their daily life who could share their experience?

I use them for research on new features. If my feature is going to interact with a frontier language model in prod, I start with these free local ones which are all competent enough to produce structured output, make tool calls, interact with mcp etc. I don’t care much for the content at the early phase of engineering, I care about the schema & failure modes.

Then when I’m getting close to feature-complete, I’ll move to a hosted frontier model for the final integration.

Cost savings are enormous if you’re making dozens of calls to language models a minute.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#173

Post by dofm »

kristjansson wrote: post #164
thomasjb wrote: post #155

Unfortunately there's no gguf quants of the assistant model yet: https://huggingface.co/models?other=base_model:quantized:goo...

I think MTP Gemma4 support is still WIP https://github.com/ggml-org/llama.cpp/pull/23398 ?

This has been my impression.

The underlying LiteRT-LM framework used in the edge gallery does support the MTP drafters for the smaller models, but according to:

https://developers.google.com/edge/litert-lm/models/gemma-4

> Note: LiteRT-LM supports E2B and E4B models today, with support for larger models coming soon.

So even Google aren't shipping MTP support for the 26B and 31B models yet.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#174

Post by thot_experiment »

djyde wrote: post #36

What are the use cases for these small models? Is there anyone using models of this scale in their daily life who could share their experience?

I don't know about this model, but the next one up, the 31B I've been using as an agentic coding assistant in OpenCode, and basically anything that's easy enough that I'd trust Sonnet to handle, I trust Gemma 4 to handle and it's been doing a great job, it surprises me positively much more often than negatively. I not infrequently run into situations where Gemma 4 fails to do the task and I switch to Opus 4.7 and it fails also.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#175

Post by dofm »

thot_experiment wrote: post #169
goobatrooba wrote: post #165

Earlier quoted context omitted.

Either Google changed the text or you editorialised it a tiny bit - just for all others that got excited, they mean 16GB V RAM. So a premium graphics card requiring a >2500€ device is the minimum to run this. Still progress, but not quite democratic yet. Weird though that Google might be cannibalising it's own AI subscription service?

I haven't tried this model yet, but I can run Gemma 31B w/ the MTP drafter in pure CPU at about 10tok/s so this should run at about 20-30tok/s on a decent CPU, it'll probably run at >50tok/s on any Mac that can fit it, and lots of people have a gaming GPU with enough VRAM. In terms of access to hardware being a gate, it's one you can hop pretty easily.

Could you outline how you are running the MTP drafters? I've tried LM Studio but no dice there. I'm probably missing something but I think llama.cpp and Ollama can't do it yet either?

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#176

Post by thot_experiment »

kristjansson wrote: post #164
thomasjb wrote: post #155

Unfortunately there's no gguf quants of the assistant model yet: https://huggingface.co/models?other=base_model:quantized:goo...

I think MTP Gemma4 support is still WIP https://github.com/ggml-org/llama.cpp/pull/23398 ?

[dead]

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#177

Post by Flere-Imsaho »

asim wrote: post #137

We are now entering the closed loop game. Google doesn't need anyone else to accelerate their models. This is their bread and butter. I'm both shocked but also not surprised that they continue to develop such efficiencies. Honestly it's like silicon and CPU architecture advancement. We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x…

Yes I've taken the "must optimise longevity" route, taking priority over other things such as my career and hobbies. I want to see the future - all this AI stuff fascinates me.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#179

Post by thot_experiment »

brianwawok wrote: post #34
nickandbro wrote: post #3

Wow Google is becoming the new pre Llama 4 Meta when it comes to releasing open weights models.

Every other Google model I have tried felt very weak compared to qwen models. I dont have a ton of use case for multimodal though, so its very possible this is a fantastic multimodal model.

Hard disagree, Qwen multimodal is way better than google's, but Gemma 31b runs laps around Qwen 27B in complex engineering tasks. Maybe Qwen is better at slopcoding web framework CRUD, but for embedded dev there's no comparison.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#180

Post by thot_experiment »

lxgr wrote: post #60

Am I missing something or are the Ollama versions of this ( https://ollama.com/library/gemma4/tags ) text-only for now?

Ollama is a shitty project that steals from the open source community, don't use it, use llama.cpp instead.
Post reply on HN