Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

211–220 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#211

Post by jimmy76615 »

ma2kx wrote: post #170
petercooper wrote: post #132

Its image processing is terrible. I ran several tests against it against Qwen 3.5 0.8b (yes, 7% the size) and Qwen beat it every time with Gemma often getting things entirely wrong. I even gave it a plain image saying "This is a test" and it thought for 6 minutes trying to analyze it and failed. Qwen 3.5 0.8b confidently got it in under a second . It may be that the Q6 quant I got is borked (or my LM Studio is), but…

I guess Google implements more / stronger guard rails than Alibaba and thus confuses these small models. At least this was my impression with Gemma3 models where it often said that the image contains some nudity / sex scenes and therefore it cannot give a description of the image. Never understood the point of this behavior....

The biggest problem with all the Google models has always been RLHF, particularly safety training. They take a good, smart model and make it behave like a corporate person that has been to far to many forced anti-{sexism, racism...} seminars so that it is now living in fear of saying something that could be construed as wrong by some moral standard.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#212

Post by magicalhippo »

kristjansson wrote: post #5
minimaxir wrote: post #2

The big story here is the encoder-free part, which I still don't fully understand. > Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. That's technically encoding, just without using a dedicated model for it like SigLIP? The Developer's Guide elaborates, it's still a 35M layer which I am curious is ro…

> quantization 12b means 12G @ 8 bits/param (basically lossless) and 6G at 4 b/p (generally accepted 'pretty close' level). Not too bad? But TBD how well the base model performs before thinking too much about quantization

Smaller models are less forgiving to quantization. For a 12B model I wouldn't expect Q4 to be "pretty close", unless it underwent quantization aware training (QAT). Of course it's not set in stone, there's a huge variance between models, so this might surprise.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#213

Post by superkuh »

senko wrote: post #120

I ran the Q4 quant (used with llama.cpp) though my "minesweeper" vibe-coding benchmark: https://senko.net/vibecode-bench/2026/minesweeper-gamma-4-12... The result is decent, but it had a few bizzare/trivial syntax errors I had to fix manually: it would do an extra closing bracket or paren a few times, and wanted to separate function definitions with comma. Not sure what that was about, but otherwise the output run ju…

>consumer-grade card with 12G of VRAM and got 5t/s

That speed for token output indicates to me that it somehow is using hybrid mode and involving cpu+system ram somehow. That ~5tk/s is about the ram bandwidth of DDR4 RAM versus that size model at 4bit. Any consumer GPU with 12 GB like a nvidia rtx 2080 or rtx 3060 should be doing 20+ tk/s with llama.cpp and CUDA backend.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#214

Post by jimmy76615 »

embedding-shape wrote: post #37
seba_dos1 wrote: post #29

Earlier quoted context omitted.

Note that a binary released under Apache 2.0 license does not yet make it FOSS.

Agreed, miles ahead though from "proprietary" which is what Meta been using for most model releases. Ideally companies would share the fucking datasets and training code already, but no, no one wants to talk about the source of those or even share the ones they have as then who knows what comes out of Pandora's box...

NVIDIA does a pretty good job on that front.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#215

Post by andai »

selectodude wrote: post #196
altruios wrote: post #104

Earlier quoted context omitted.

Tokens are such a strange base unit. Couldn't we do something that naturally conforms better to reality than such choppy units that cause all sorts of artifacts? making everything 'language based' prevents true multi-modality. Thinking isn't done in language. Thinking outputs language, but its far more like multiple waves of data coalescing into an 'idea', internal... subjectively (n=1) at least. I think wave/signal…

Not to be too snarky but there’s a few trillion dollars and some of the brightest minds of our generation working on this. I’m sure there’s a reason why they’ve settled for or are stuck on tokenization.

Yeah, I'm sure we ended up with JavaScript for great reasons too.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#217

Post by echelon »

airstrike wrote: post #116
echelon wrote: post #91

Earlier quoted context omitted.

Google needs an anti-trust breakup about 10 years ago. They need one more than ever now. This is ridiculously anti-competitive.

This is literally competition

1. Google is dumping on the market to weaken OpenAI and Anthropic.

2. Every time you search for Claude or ChatGPT, you get presented with an AdWords bidding war.

3. Google is deploying its models in Search, Docs/Drive/Office, YouTube, Chrome, ...

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#218

Post by CamperBob2 »

refulgentis wrote: post #200
altruios wrote: post #104

Earlier quoted context omitted.

Tokens are such a strange base unit. Couldn't we do something that naturally conforms better to reality than such choppy units that cause all sorts of artifacts? making everything 'language based' prevents true multi-modality. Thinking isn't done in language. Thinking outputs language, but its far more like multiple waves of data coalescing into an 'idea', internal... subjectively (n=1) at least. I think wave/signal…

This sounds like when crystal people talk quantum physics.

I agree with the GP. The idea that there's not a better intermediate representation between tokens and embedding vectors seems absurd. But how to arrive at such a representation and implement it effectively is a few zeroes above my pay grade.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#219

Post by airstrike »

echelon wrote: post #217
airstrike wrote: post #116

Earlier quoted context omitted.

This is literally competition

1. Google is dumping on the market to weaken OpenAI and Anthropic. 2. Every time you search for Claude or ChatGPT, you get presented with an AdWords bidding war. 3. Google is deploying its models in Search, Docs/Drive/Office, YouTube, Chrome, ...

1. This isn't dumping

2. I'm not sure what this has to do with the case, unless you're arguing Google has an ads monopoly, in which case the best argument would likely not be that adwords lead to bidding wars because that just sounds like they're selling a product people really want to pay for

3. There's nothing criminal about being a very diversified business

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#220

Post by dofm »

thot_experiment wrote: post #207
dofm wrote: post #175

Earlier quoted context omitted.

Could you outline how you are running the MTP drafters? I've tried LM Studio but no dice there. I'm probably missing something but I think llama.cpp and Ollama can't do it yet either?

I just build llama.cpp from scratch on the PR that has MTP drafters. https://github.com/ggml-org/llama.cpp/pull/23398 Please don't use Ollama, it's a bad actor in the OSS community.

I don't have the energy to build stuff all the time, that's a rabbit-hole side tunnel I don't really want to get into. I have larger concerns in my life that are more urgent than developing that side of things.

But I've moved on from Ollama for the time being, though I am mainly interested to see what the Gemma 4 MTP speeds are like on my M1 Max, so I may test it.

I am quite impressed with the tools in LM Studio, which is also a beautiful app, but it is not open source (which challenges my personal strategy somewhat) and I dread its inevitable enshittification.

Nevertheless the GUI has been very helpful while I learn, and I will probably use it until something else presents or my usage pattern settles down from experimentation to something a bit more routine.

I will try oMLX, too, but judging by the LiteRT page I may soon be able to just use that for the larger models if I end up settling with Gemma 4.

Post reply on HN