Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

301–310 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#301

Post by sureglymop »

dgacmu wrote: post #291

I was excited about this until I fed it one of my local test problems: coin identification. I then spent 10 minutes arguing with it that a photo of a 1998 washington quarter was not, in fact, a Morgan Silver Dollar. I mean, I wish it was. It went into a crash loop on a british columbia 1 dollar coin. This happened with both Q4_1 and Q8. Maybe I'm holding it wrong or it's just really bad for this task. In contrast, ge…

Why would you expect it to be good for this particular highly specific task? Curious.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#302

Post by sureglymop »

djyde wrote: post #36

What are the use cases for these small models? Is there anyone using models of this scale in their daily life who could share their experience?

I moreso run other small special purpose models like Whisper, SAM, Matcha, CLIP etc. and then do contextual correction passes with models like this.

Think almost like unix pipelines, have used it for many workflows.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#303

Post by dgacmu »

sureglymop wrote: post #301
dgacmu wrote: post #291

I was excited about this until I fed it one of my local test problems: coin identification. I then spent 10 minutes arguing with it that a photo of a 1998 washington quarter was not, in fact, a Morgan Silver Dollar. I mean, I wish it was. It went into a crash loop on a british columbia 1 dollar coin. This happened with both Q4_1 and Q8. Maybe I'm holding it wrong or it's just really bad for this task. In contrast, ge…

Why would you expect it to be good for this particular highly specific task? Curious.

Ah! Good question: Google's non-open-weights models (Gemini, etc) have almost always outperformed on image recognition tasks compared to any other models. I use a mix of in-house and Gemini for image classification tasks for $startup. No other models have done as well, and I had hoped that some of that would spill over into their open source models. It does to a degree - bigger Gemma models are okay.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#304

Post by lambda »

lambda wrote: post #159
matja wrote: post #43

Earlier quoted context omitted.

One side-effect, is that the separate .mmproj file (Multi-Modal Projection encoder) is no longer needed, when using the model with llama.cpp etc.

It's not? There's an mmproj in the GGUFs released by ggml-org: https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/mai... From the visual guide, there's still the 35M parameter embedder, then the linear projector, for vision, and the linear projector for audio, so it does have some parameters used for the multimodal input to project it into the LLM latent space: https://newsletter.maartengrootendorst.com/p/a-vis…

Ah, Unsloth has uploaded mmproj now as well.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#305

Post by hedgehog »

SwellJoe wrote: post #296
zozbot234 wrote: post #292

Earlier quoted context omitted.

Why does everyone expect interactivity from local AI? It's not the best use of the hardware, especially not miniPC hardware. Long-term batched inference with larger and more capable models is much more feasible AIUI.

Even batched it's uncomfortably slow. I started to benchmark ds4 with my security vulnerability benchmark (after Qwen 3.6 dense and MoE and a bunch of cloud models), but it was going to tie up the Strix Halo for more than a day, so I decided not to run it as it would prevent me from doing other stuff with it during that time. Even batched usage needs to be fast enough to deliver results in a reasonable time. Overnigh…

The 6-bit versions + 8-bit KV cache seems to save a good bit of memory without a significant loss of quality. The Qwen 35B is pretty fast in my testing, but MiniMax M2.7 230B is in some ways faster (way fewer tokens to arrive at an answer) even though it is much larger.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#306

Post by btbuildem »

asim wrote: post #137

We are now entering the closed loop game. Google doesn't need anyone else to accelerate their models. This is their bread and butter. I'm both shocked but also not surprised that they continue to develop such efficiencies. Honestly it's like silicon and CPU architecture advancement. We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x…

> which of anyone is religious knows Noah and others lived to that age in a totally different era

My favourite conspiracy theory lately is that the above isn't a silly fairy tale, that we actually used to live much much longer -- until the common cold came on the scene, and the sequelae dramatically shortened our lifespans. Today we dismiss it as "just a cold" unbeknownst of what it robbed us from.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#307

Post by SwellJoe »

hedgehog wrote: post #305
SwellJoe wrote: post #296

Earlier quoted context omitted.

Even batched it's uncomfortably slow. I started to benchmark ds4 with my security vulnerability benchmark (after Qwen 3.6 dense and MoE and a bunch of cloud models), but it was going to tie up the Strix Halo for more than a day, so I decided not to run it as it would prevent me from doing other stuff with it during that time. Even batched usage needs to be fast enough to deliver results in a reasonable time. Overnigh…

The 6-bit versions + 8-bit KV cache seems to save a good bit of memory without a significant loss of quality. The Qwen 35B is pretty fast in my testing, but MiniMax M2.7 230B is in some ways faster (way fewer tokens to arrive at an answer) even though it is much larger.

Qwen 3.6 35B-A3B with MTP at 8 bits is blazing fast, something like 50-60 tokens per second. That's plenty fast for interactive use, so I haven't tried lower bits. Unfortunately the MoE is notably dumber than the dense model (for the case I have data about...I've been benchmarking models for security vulnerability scanning, and 27B is notably better on hard bugs).

The dense model is almost usable, but feels really sluggish, even with MTP. I think it's about 12-15 tokens/second on the Strix Halo. Slow enough to where I'd rather pay to use a cloud model.

I might try the 6-bit version of the dense model to see how it behaves, though. Maybe it'll retain its bug hunting abilities while making it fast enough to use interactively and not take all day for benchmark runs.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#308

Post by hattimaTim »

bitexploder wrote: post #263
ActorNightly wrote: post #227

Earlier quoted context omitted.

Google is an advertising company first and foremost. At some point, these local models have to fit into that umbrella. I don't quite know how yet, but its going to happen. That being said, the real value in paid plans is that you get ecosystem integration that can read your gmail, photos, docs, and so on.

Google is also a Cloud Provider. Cloud is now ~18% of Google. While it is an advertising juggernaut. Cloud is also rapidly growing, so the local models simply fit as AI research and dev and getting more people on Gemini models. They /are/ advertising, effectively :)

I wish they were :) But the gemini models are so unstable in API that I can not even use them for production.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#309

Post by sourcecodeplz »

dotancohen wrote: post #273
0xbadcafebee wrote: post #206

Earlier quoted context omitted.

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

> For 16GB laptops, Qwen 3.5 9B is the undisputed champ. You seem like the guy to ask. For a laptop with 12GB VRAM (RTX 5070) and 32 GB system RAM, what is a good multilingual (English, Hebrew, Greek) model for conversing with personal notes in Org mode format? I don't care how long updating the model or rag takes, and even inference can be reasonably slow, but the results of the query as they relate to my personal n…

Any Gemma 4 model, they are great at translations, multilingual

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#310

Post by jmpeax »

0xbadcafebee wrote: post #206
senko wrote: post #120

I ran the Q4 quant (used with llama.cpp) though my "minesweeper" vibe-coding benchmark: https://senko.net/vibecode-bench/2026/minesweeper-gamma-4-12... The result is decent, but it had a few bizzare/trivial syntax errors I had to fix manually: it would do an extra closing bracket or paren a few times, and wanted to separate function definitions with comma. Not sure what that was about, but otherwise the output run ju…

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

> nowhere in the announcement is coding mentioned

It's right there in the middle benchmark bar "LiveCode Bench" 72%.

Post reply on HN