Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

311–320 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#311

Post by emmelaich »

dotancohen wrote: post #273
0xbadcafebee wrote: post #206

Earlier quoted context omitted.

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

> For 16GB laptops, Qwen 3.5 9B is the undisputed champ. You seem like the guy to ask. For a laptop with 12GB VRAM (RTX 5070) and 32 GB system RAM, what is a good multilingual (English, Hebrew, Greek) model for conversing with personal notes in Org mode format? I don't care how long updating the model or rag takes, and even inference can be reasonably slow, but the results of the query as they relate to my personal n…

You may like https://www.llmfit.org/

(not recommendation, I've not used it .. yet)

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#312

Post by Scrounger »

asim wrote: post #137

We are now entering the closed loop game. Google doesn't need anyone else to accelerate their models. This is their bread and butter. I'm both shocked but also not surprised that they continue to develop such efficiencies. Honestly it's like silicon and CPU architecture advancement. We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x…

> May God protect us.

Today, data systems and algorithms can be deployed at unprecedented scale and speed. Unintended consequences will affect people with that same scale and speed

—Michael Chapman

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#313

Post by hedgehog »

SwellJoe wrote: post #307
hedgehog wrote: post #305

Earlier quoted context omitted.

The 6-bit versions + 8-bit KV cache seems to save a good bit of memory without a significant loss of quality. The Qwen 35B is pretty fast in my testing, but MiniMax M2.7 230B is in some ways faster (way fewer tokens to arrive at an answer) even though it is much larger.

Qwen 3.6 35B-A3B with MTP at 8 bits is blazing fast, something like 50-60 tokens per second. That's plenty fast for interactive use, so I haven't tried lower bits. Unfortunately the MoE is notably dumber than the dense model (for the case I have data about...I've been benchmarking models for security vulnerability scanning, and 27B is notably better on hard bugs). The dense model is almost usable, but feels really sl…

Same chip, with a 6 bit 35B and 8 bit KV cache I see about 500 prefill and 55 decode at 30k into the context window. MiniMax seemed a bit lower token rate but much, much less prone to 40k tokens of monologue before generating an answer. A pattern I like is to use a smaller model to do most execution and then a larger model to review transcripts and output and do any fixups and tooling improvements (this is all batch jobs so all I care about is overall throughput).

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#315

Post by int_19h »

zozbot234 wrote: post #292
SwellJoe wrote: post #285

Earlier quoted context omitted.

Yep, I have a Strix Halo and while it can run models bigger than Qwen 3.6 27b, it's not usable interactively when you do. ds4 patched for ROCm works, but at such a slow speed, it's not usable for coding agents. The Nvidia boxes have only slightly more memory bandwidth, so I wouldn't expect them to be notably faster. At least not enough to make it useful interactively at that scale.

Why does everyone expect interactivity from local AI? It's not the best use of the hardware, especially not miniPC hardware. Long-term batched inference with larger and more capable models is much more feasible AIUI.

I can't speak for others but IMO the only reason to run models locally right now is privacy - i.e. you don't trust any of the cloud providers to not look at your prompts. Price-wise the market is extremely competitive and cheap model serving favors large scale so anything that can be run locally can be run cheaper in the cloud. But if privacy is important, then it's important for everything, including traditional chatbot applications, which kinda do require interactivity.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#316

Post by kajecounterhack »

zozbot234 wrote: post #287
MoonWalk wrote: post #277

Earlier quoted context omitted.

Why the obfuscation in the first place?

I'm not sure that GP is correct, many people in that forum tend to hate Qwen for closing up many of their more recent models and leaving the whole local inference community 'stranded' on their older releases.

That was definitely not the subreddit where I got my info.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#318

Post by browningstreet »

johnnyApplePRNG wrote: post #105
browningstreet wrote: post #23

Earlier quoted context omitted.

This won't replace commercially viable, revenue generating alternatives of their own devising, but it does enable development activity and initiate conversations with enterprises who start with this model but want to do slightly more. That's my experience right now... my company is all in on a plethora of platform products. Also, Microsoft just yesterday said their goal was "Unmetered intelligence". There's a lot of…

re "Unmetered intelligence" goal of Microshaft. Of course it is... This is Windows-Licensing-Level Money Opportunity 2.0.

I said they “said” that.

And Google releases another free local model. As did Microsoft.

The actual facts of the day belie your snort take. At least a little bit.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#319

Post by julianlam »

zozbot234 wrote: post #287
MoonWalk wrote: post #277

Earlier quoted context omitted.

Why the obfuscation in the first place?

I'm not sure that GP is correct, many people in that forum tend to hate Qwen for closing up many of their more recent models and leaving the whole local inference community 'stranded' on their older releases.

Are you sure? Prior to today the sub seems to be pretty partial to Qwen.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#320

Post by KiwiJohnno »

madduci wrote: post #140
minimaxir wrote: post #2

The big story here is the encoder-free part, which I still don't fully understand. > Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. That's technically encoding, just without using a dedicated model for it like SigLIP? The Developer's Guide elaborates, it's still a 35M layer which I am curious is ro…

VRAM, not RAM. I wish it was light enough for iGPUs too

I ran the 26B model on my i5 which has no discrete graphics card. It ran about 7 tokens/sec, and appreared to be a very capable model.
Post reply on HN