Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

181–190 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#181

Post by thot_experiment »

kylehotchkiss wrote: post #124
RandyOrion wrote: post #111

A small dense multimodal model with audio support, interesting. Wait, *Excluding Chinese language. This is ... curious. P.S. Where is gemma 4 124b?

Where are the computers we could purchase to run 124b models :’(

You can get SXM V100s for like $100 off ebay, if you're willing to do the troubleshooting work to get em running with adapters you can build a computer capable of fitting a Q4 quant of a 120b model in VRAM for something like fifteen hundred dollars. (assuming you already have some RAM sticks laying around T___T)

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#182

Post by scirob »

Quickly deployed it to check some benchmarks relevant for German language. These are results for CohereLabs/include-base-44 german only : Gemma 4 12B %61.9

  Gemma 4 26B (a4b MoE)    0.647
  Qwen 3 14B               0.621 
  Gemma 4 12B              0.618
  Ministral 14B 2512       0.604 
  Gemma 3 12B              0.547
The quwen 3 14B vs Gemma 4 12B difference is within random variance they same in some repeat runs they actually got the exact same score. Next step up Gemma 4 31B gets 0.676 on this. Or let in some reasoning Qwen 3 14B (reasoning) 0.676.

I'll run some cheat-proof benchmarks ones tomorrow see if qwen is still on top.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#183

Post by schipperai »

ethanpil wrote: post #13

What's Google's business case for releasing open models? Don't get me wrong, I am grateful and appreciative of these releases. I'm trying to understand how it fits into their bigger picture as a for profit company? Are they not helping competitors build on the novel technology they have developed? Is it simply goodwill and/or marketing? Or am I missing something strategic?

Demis at YCombinator said that they think its best their edge models are open cause once they are put on device they are vulnerable anyways

https://youtu.be/JNyuX1zoOgU?is=PdzCILyi8SP6cfDr

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#184

Post by Pixel-Labs »

WhitneyLand wrote: post #117
GaggiX wrote: post #4

Earlier quoted context omitted.

> That's technically encoding Isn't that just projecting the patches into the d_model size vectors that the models takes? >I am assuming that involves of quantization 12B model in 16GB seems very reasonable to me, int8 is top quality for running models.

I don’t think so, the HF weights are bf16 which means 24GB + cache/overhead. It sounds like marketing spin where the performance claims are based on BF16 and the “runs in 16GB” claim is on a totally different quantized version.

[flagged]

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#185

Post by christina97 »

It seems worse in all aspects to the 26B A4B? I would have thought dense models beat MoE still on many benchmarks?

Is the entire point of this model then that it runs if you don’t have enough GPU memory to load the 26B? That one runs faster anyway due to lower active params.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#187

Post by zigzag312 »

senko wrote: post #120

I ran the Q4 quant (used with llama.cpp) though my "minesweeper" vibe-coding benchmark: https://senko.net/vibecode-bench/2026/minesweeper-gamma-4-12... The result is decent, but it had a few bizzare/trivial syntax errors I had to fix manually: it would do an extra closing bracket or paren a few times, and wanted to separate function definitions with comma. Not sure what that was about, but otherwise the output run ju…

> It roughly compares with GPT-4.1 (!!), released 14 months ago

I think the mayor win for coding was reasoning. That's why such a small model can match GPT-4.1 in coding, but I suspect that GPT-4.1 still wins in general world knowledge due to bigger size.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#188

Post by Patrick_Devine »

dofm wrote: post #175
thot_experiment wrote: post #169

Earlier quoted context omitted.

I haven't tried this model yet, but I can run Gemma 31B w/ the MTP drafter in pure CPU at about 10tok/s so this should run at about 20-30tok/s on a decent CPU, it'll probably run at >50tok/s on any Mac that can fit it, and lots of people have a gaming GPU with enough VRAM. In terms of access to hardware being a gate, it's one you can hop pretty easily.

Could you outline how you are running the MTP drafters? I've tried LM Studio but no dice there. I'm probably missing something but I think llama.cpp and Ollama can't do it yet either?

I haven't yet pushed the MTP enabled gemma4 12b model for Ollama because in my testing I wasn't getting a performance bump. The other gemma4 MTP models should work OK right now, but there are some fixes we're just about to push. This is specifically for the MLX backend.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#189

Post by ch_sm »

dofm wrote: post #175
thot_experiment wrote: post #169

Earlier quoted context omitted.

I haven't tried this model yet, but I can run Gemma 31B w/ the MTP drafter in pure CPU at about 10tok/s so this should run at about 20-30tok/s on a decent CPU, it'll probably run at >50tok/s on any Mac that can fit it, and lots of people have a gaming GPU with enough VRAM. In terms of access to hardware being a gate, it's one you can hop pretty easily.

Could you outline how you are running the MTP drafters? I've tried LM Studio but no dice there. I'm probably missing something but I think llama.cpp and Ollama can't do it yet either?

can‘t speak to compatibility with this new model, but oMLX supports MTP drafters very well.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#190

Post by Zambyte »

Zambyte wrote: post #32

Is this Mac only? Or is that an Ollama issue that it only supports this release of models on Mac? It seems like every tag with the MLX badge is only supported on Mac[0], and that includes all of the tags in this release. [0] https://ollama.com/library/gemma4/tags Edit: MLX being Mac-only is independent of the model being MLX (and therefore Mac) only. The latter is what I am asking about.

The non-MLX versions just dropped on Ollama. gemma4:12b-it-q8_0, gemma4:12b-it-bf16, etc.
Post reply on HN