Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

281–290 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#281

Post by iso1631 »

0xbadcafebee wrote: post #206
senko wrote: post #120

I ran the Q4 quant (used with llama.cpp) though my "minesweeper" vibe-coding benchmark: https://senko.net/vibecode-bench/2026/minesweeper-gamma-4-12... The result is decent, but it had a few bizzare/trivial syntax errors I had to fix manually: it would do an extra closing bracket or paren a few times, and wanted to separate function definitions with comma. Not sure what that was about, but otherwise the output run ju…

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

I find ram crazy. My thinkpad has 32G of ram, it's a t470 that's nearly a decade old

Why do people with modern laptops have such little amounts of ram?

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#282

Post by pseudollm »

pseudosavant wrote: post #259
senko wrote: post #120

I ran the Q4 quant (used with llama.cpp) though my "minesweeper" vibe-coding benchmark: https://senko.net/vibecode-bench/2026/minesweeper-gamma-4-12... The result is decent, but it had a few bizzare/trivial syntax errors I had to fix manually: it would do an extra closing bracket or paren a few times, and wanted to separate function definitions with comma. Not sure what that was about, but otherwise the output run ju…

Models this small and this capable bode really well for the usefulness of a PC like the RTX Spark that Nvidia/Microsoft announced this week. 128GB of unified memory will likely be more than sufficient for effective local agentic coding, even if SOTA cloud models will still be even better. Up until this point, I've found the cost/value to unequivocally favor using a cloud subscription, but I would be lying if I didn't…

> usefulness of the RTX Spark

Not really. There's a reason the announcement didn't include ANY benchmark (!) and didn't mention EXACTLY what is the memory bandwidth. It's going to be dog-slow unusable for large models, as tok/sec is basically bandwidth divided by active weights. Rumoured 300GB/s / 30GB active weights (decent model) = 10 tokens per second, which is really slow

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#283

Post by Aachen »

jack_pp wrote: post #280
Aachen wrote: post #257

Earlier quoted context omitted.

Arguably, but that's not the point. Take image (e.g. png) files on a CD-ROM shipped by a game vendor, which can be trivially copied even by my grandma. That doesn't move the game vendor to release them as freely distributable under the Apache license

Good point but still, why would Google police this model? If they had a restrictive licence on it do you think it would be worth it for them to enforce it? This way they at least buy some good will and mindshare

That makes sense to me. Guess one might say the same for game icons and other such files that lay around in disks, but yeah maybe it's as simple as that

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#284

Post by pseudollm »

make3 wrote: post #84
mchinen wrote: post #72

Earlier quoted context omitted.

The audio side is even more interesting, as it seems they totally got rid of positional embedding are just doing a single linear transform to match the LLM input dimension and that's it. > Audio: We simplified audio processing even further. We removed the audio encoder entirely and projected the raw audio signal into the same dimensional space as text tokens.

I guarantee you there's positional information one way or another. they just don't mention it because positional embeddings are extremely cheap computationally, not worth mentioning

No there isn't - read the paper. It's just 40msec raw audio samples. Multiplied by one matrix to translate to 3800 input vector. That's it. The next 40 msec are fed in the next transformer input step. Without any positional encoding. Repeat ad infinitum

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#285

Post by SwellJoe »

pseudollm wrote: post #282
pseudosavant wrote: post #259

Earlier quoted context omitted.

Models this small and this capable bode really well for the usefulness of a PC like the RTX Spark that Nvidia/Microsoft announced this week. 128GB of unified memory will likely be more than sufficient for effective local agentic coding, even if SOTA cloud models will still be even better. Up until this point, I've found the cost/value to unequivocally favor using a cloud subscription, but I would be lying if I didn't…

> usefulness of the RTX Spark Not really. There's a reason the announcement didn't include ANY benchmark (!) and didn't mention EXACTLY what is the memory bandwidth. It's going to be dog-slow unusable for large models, as tok/sec is basically bandwidth divided by active weights. Rumoured 300GB/s / 30GB active weights (decent model) = 10 tokens per second, which is really slow

Yep, I have a Strix Halo and while it can run models bigger than Qwen 3.6 27b, it's not usable interactively when you do. ds4 patched for ROCm works, but at such a slow speed, it's not usable for coding agents.

The Nvidia boxes have only slightly more memory bandwidth, so I wouldn't expect them to be notably faster. At least not enough to make it useful interactively at that scale.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#286

Post by zozbot234 »

pseudosavant wrote: post #259
senko wrote: post #120

I ran the Q4 quant (used with llama.cpp) though my "minesweeper" vibe-coding benchmark: https://senko.net/vibecode-bench/2026/minesweeper-gamma-4-12... The result is decent, but it had a few bizzare/trivial syntax errors I had to fix manually: it would do an extra closing bracket or paren a few times, and wanted to separate function definitions with comma. Not sure what that was about, but otherwise the output run ju…

Models this small and this capable bode really well for the usefulness of a PC like the RTX Spark that Nvidia/Microsoft announced this week. 128GB of unified memory will likely be more than sufficient for effective local agentic coding, even if SOTA cloud models will still be even better. Up until this point, I've found the cost/value to unequivocally favor using a cloud subscription, but I would be lying if I didn't…

RTX Spark is pretty much the DGX Spark in a laptop form factor, plus some lower-performing chips in the same series to be released later according to rumors. We know quite well how the top-of-the-line chip performs: it's very interesting for some application areas, less so for others.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#287

Post by zozbot234 »

MoonWalk wrote: post #277
xenophonf wrote: post #268

Earlier quoted context omitted.

It took me way too long to realize you were referring to r/localllama.

Why the obfuscation in the first place?

I'm not sure that GP is correct, many people in that forum tend to hate Qwen for closing up many of their more recent models and leaving the whole local inference community 'stranded' on their older releases.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#288

Post by 0xbadcafebee »

asim wrote: post #137

We are now entering the closed loop game. Google doesn't need anyone else to accelerate their models. This is their bread and butter. I'm both shocked but also not surprised that they continue to develop such efficiencies. Honestly it's like silicon and CPU architecture advancement. We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x…

1996 didn't look that different than today, in the US anyway. Biggest difference, besides the electric cars, is everybody has a phone but nobody uses it to talk to people.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#289

Post by superkuh »

senko wrote: post #226
superkuh wrote: post #213

Earlier quoted context omitted.

>consumer-grade card with 12G of VRAM and got 5t/s That speed for token output indicates to me that it somehow is using hybrid mode and involving cpu+system ram somehow. That ~5tk/s is about the ram bandwidth of DDR4 RAM versus that size model at 4bit. Any consumer GPU with 12 GB like a nvidia rtx 2080 or rtx 3060 should be doing 20+ tk/s with llama.cpp and CUDA backend.

Good catch. I haven't looked deeply into it. This is with Vulkan backend on Linux which I understand should be roughly comparable to CUDA? Gfx is rtx 3060(ti?). I should play a bit more with llama.cpp options and see what bappened there. Thanks!

I've had it happen in the past with llama.cpp on linux that the CPU will present itself as a vulkan device GPU1 with "PHYSICAL_DEVICE_TYPE_CPU" and had a mix-up. Might want to try llama-server --list-devices and then append --device Vulkan0 or whatever.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#290

Post by mitkebes »

ActorNightly wrote: post #234
asim wrote: post #137

We are now entering the closed loop game. Google doesn't need anyone else to accelerate their models. This is their bread and butter. I'm both shocked but also not surprised that they continue to develop such efficiencies. Honestly it's like silicon and CPU architecture advancement. We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x…

Nope, lol. Large models still are quite far ahead, don't be fooled that even Gemma:31b (which is better than the 12b overall) is anywhere close to big models. There is definitely room for optimization, but fundamentally, for complex tasks, you need visible small gradients for accuracy that allow the model to be trained on (and consequently be followed during inference). For example, if you specify in instructions not…

It doesn't matter if Large models are undeniably better, if a local model is "good enough" to handle the task. With API costs ramping up, I think a lot of companies are going to want to look into what can be run locally instead, possibly only using larger models when the local models fall short.
Post reply on HN