Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

261–270 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#261

Post by Patrick_Devine »

embedding-shape wrote: post #248
Patrick_Devine wrote: post #191

Earlier quoted context omitted.

I realize this is a little confusing; we're working w/ the MLX team to bring MLX to other platforms, but we're not quite there yet. The `gemma4:12b-nvfp4` model is specifically for the MLX engine. For the GGUF 4bit variant (i.e. non-macs) you'll need `gemma4:12b-it-q4_K_M` which I just pushed. You'll also need to upgrade to version 0.30.4 which we're just about to release (it's in prerelease and we're running through…

I gotta say, having both "gemma4:12b-mlx-bf16" and "gemma4:12b-nvfp4" be MLX-specific, and not labeling all of the MLX-specific ones as such, is a bit different than "little confusing" and more "set up to be confusing" :) > You'll also need to upgrade to version 0.30.4 which we're just about to release Interesting, wasn't Google coordinating today's release with you? Considering the blog post seems to have gone out w…

Given the model was just republished by Google 15 minutes ago and we're going to have to redo everything (and everyone will have to redownload for all platforms -- not just Ollama), I'll just say that sometimes things don't work out exactly the way you want them to. :-D

That said, I think the gemma4:12b-nvfp4 model is pretty solid. It's been tuned with Nvidia's model optimizer. I've been waiting on the results for MMLU-Pro, but I'll have to retrigger that after reconverting.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#262

Post by bitexploder »

mdp2021 wrote: post #208
zigzag312 wrote: post #187

Earlier quoted context omitted.

> It roughly compares with GPT-4.1 (!!), released 14 months ago I think the mayor win for coding was reasoning. That's why such a small model can match GPT-4.1 in coding, but I suspect that GPT-4.1 still wins in general world knowledge due to bigger size.

> I suspect ... still wins in general world knowledge due to bigger size Encyclopedic knowledge matters relatively little in perspective, given the expectable future developments: even the more knowledgeable of us will use that knowledge for reasoning and intuition (and we will have absorbed the intellectual keys during our training), but under our professional hat we should in theory be ready to go "I stand correcte…

Don't LLMs work on attention though? The closer in their hyperdimensional space you can land your problem to their inherent understand the better they are at understanding your problem domain. RAG loops can be very slow and agents may simply lack the knowledge to use them correctly.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#263

Post by bitexploder »

ActorNightly wrote: post #227
goobatrooba wrote: post #165

Earlier quoted context omitted.

Either Google changed the text or you editorialised it a tiny bit - just for all others that got excited, they mean 16GB V RAM. So a premium graphics card requiring a >2500€ device is the minimum to run this. Still progress, but not quite democratic yet. Weird though that Google might be cannibalising it's own AI subscription service?

Google is an advertising company first and foremost. At some point, these local models have to fit into that umbrella. I don't quite know how yet, but its going to happen. That being said, the real value in paid plans is that you get ecosystem integration that can read your gmail, photos, docs, and so on.

Google is also a Cloud Provider. Cloud is now ~18% of Google. While it is an advertising juggernaut. Cloud is also rapidly growing, so the local models simply fit as AI research and dev and getting more people on Gemini models. They /are/ advertising, effectively :)

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#264

Post by altruios »

TeMPOraL wrote: post #241
altruios wrote: post #104

Earlier quoted context omitted.

Tokens are such a strange base unit. Couldn't we do something that naturally conforms better to reality than such choppy units that cause all sorts of artifacts? making everything 'language based' prevents true multi-modality. Thinking isn't done in language. Thinking outputs language, but its far more like multiple waves of data coalescing into an 'idea', internal... subjectively (n=1) at least. I think wave/signal…

> making everything 'language based' prevents true multi-modality. Thinking isn't done in language. Thinking outputs language Your problem isn't with tokens, but with "language". Tokens have little to do with language, other than usually being consumed in sequence, but that's true of anything that has to span over time. Thinking of tokens as letters or subwords is mistaking the general with the specific. We may have…

Can you elaborate more on what a token looks like as a pixel patch/sound/general signal as it currently is (in this model)?

My understanding of pixel representation is: slice a grid in an image, each square slice gets projected into a number array of x long (not sure how long x is, or if it's variable), which then gets projected down to a token representing that space (3-4 long as alpha-numeric) and AGAIN gets passed into "position detector" which outputs a token representing that pixel/position. which gets passed into the lmm (at a significantly reduced/translated signal into token space).

First, before continuing: do I have that mostly correct?

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#265

Post by jpadkins »

ActorNightly wrote: post #227
goobatrooba wrote: post #165

Earlier quoted context omitted.

Either Google changed the text or you editorialised it a tiny bit - just for all others that got excited, they mean 16GB V RAM. So a premium graphics card requiring a >2500€ device is the minimum to run this. Still progress, but not quite democratic yet. Weird though that Google might be cannibalising it's own AI subscription service?

Google is an advertising company first and foremost. At some point, these local models have to fit into that umbrella. I don't quite know how yet, but its going to happen. That being said, the real value in paid plans is that you get ecosystem integration that can read your gmail, photos, docs, and so on.

local models still need information retrieval.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#266

Post by thangalin »

kajecounterhack wrote: post #229
0xbadcafebee wrote: post #206

Earlier quoted context omitted.

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

Have you found Gemma 4 31B better than Qwen 3.6 27B Q8? I just started using Qwen + Pi agent and it's great, but "which model works best" is still totally crowdsourced and I was going off of peoples' opinions on reddit. Would love to hear more opinions if people have them.

Yes. I'm using Gemma-4 31B (gemma-4-31B-it-assistant.Q4_K_M.gguf) with llama.cpp to attribute quotations throughout chapters of my sci-fi novel. I started with Qwen3, but couldn't get it to work. Qwen3 TTS Voice Design, on the other hand, is incredible (Qwen3-TTS-12Hz-1.7B-VoiceDesign). I'm using both for an audiobook generator that produces a variety of voices.

Screens:

* https://i.ibb.co/TBBV5nJk/kl-01.png (voice design)

* https://i.ibb.co/nNvvKDyV/kl-02.png (quotation attributions)

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#267

Post by embedding-shape »

kajecounterhack wrote: post #229
0xbadcafebee wrote: post #206

Earlier quoted context omitted.

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

Have you found Gemma 4 31B better than Qwen 3.6 27B Q8? I just started using Qwen + Pi agent and it's great, but "which model works best" is still totally crowdsourced and I was going off of peoples' opinions on reddit. Would love to hear more opinions if people have them.

> Have you found Gemma 4 31B better than Qwen 3.6 27B Q8?

Which quant of Gemma? For coding Qwen seems to be pretty far ahead, but generally Gemma seems to have a "vaster" set of knowledge, but armed with a search tool it doesn't really matter, and Qwen 3.6 been really great for all sorts of tool calling. I mostly do programming and related things though, fwiw.

> I was going off of peoples' opinions on reddit

It's extremely astroturfed all over the place, especially the larger subreddits, and especially the one related to a specific animal in a specific location. It's sad, as early on it was a great resource, but now it's mostly paid posts and a race to the bottom, with lots of piling, and all the knowledgeable people I used to recognize are nowhere to be found.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#268

Post by xenophonf »

embedding-shape wrote: post #267
kajecounterhack wrote: post #229

Earlier quoted context omitted.

Have you found Gemma 4 31B better than Qwen 3.6 27B Q8? I just started using Qwen + Pi agent and it's great, but "which model works best" is still totally crowdsourced and I was going off of peoples' opinions on reddit. Would love to hear more opinions if people have them.

> Have you found Gemma 4 31B better than Qwen 3.6 27B Q8? Which quant of Gemma? For coding Qwen seems to be pretty far ahead, but generally Gemma seems to have a "vaster" set of knowledge, but armed with a search tool it doesn't really matter, and Qwen 3.6 been really great for all sorts of tool calling. I mostly do programming and related things though, fwiw. > I was going off of peoples' opinions on reddit It's ext…

It took me way too long to realize you were referring to r/localllama.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#269

Post by kristianp »

What quantisation do the creators intend this to be run at? They talk about 16GB of ram, so should it be run at 8 bit? People here are talking about using q4, but I would have thought a smaller model like this wouldn't perform well at such low bits per parameter. Edit, it looks like their bechmarks would have been done at 16 bit float, as the hugging face release is that size: https://huggingface.co/google/gemma-4-12B . Which is a little deceptive: they're advertising an 8 bit size will fit on 16GB laptops, while releasing a 16bit size.

I guess we have to wait for someone to produce perplexity curves at different Q's.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#270

Post by CMay »

petercooper wrote: post #132

Its image processing is terrible. I ran several tests against it against Qwen 3.5 0.8b (yes, 7% the size) and Qwen beat it every time with Gemma often getting things entirely wrong. I even gave it a plain image saying "This is a test" and it thought for 6 minutes trying to analyze it and failed. Qwen 3.5 0.8b confidently got it in under a second . It may be that the Q6 quant I got is borked (or my LM Studio is), but…

For Qwen 3.5 0.8B presumably you're running it unquantized, because it's so small. Get at least the Q8 of Gemma 4 12B with the F32 mmproj and use an f16 kv cache.

Then run it with the latest llama.cpp that contains the Gemma 4 12B unified bug fixes, using --image-min-tokens 560 --image-max-tokens 2240 --batch-size 4096 --ubatch-size 4096 --temp 1.0 --top-p 0.95 --top-k 64 --jinja

It's understanding far more complex things for me and can reliably handle tiny text, so it should be easily understanding an image that only contains the text "This is a test".

Post reply on HN