Cool!
Gemma 4 12B: A unified, encoder-free multimodal model
121–130 of 421 posts
Re: Gemma 4 12B: A unified, encoder-free multimodal model
#122What's Google's business case for releasing open models? Don't get me wrong, I am grateful and appreciative of these releases. I'm trying to understand how it fits into their bigger picture as a for profit company? Are they not helping competitors build on the novel technology they have developed? Is it simply goodwill and/or marketing? Or am I missing something strategic?
Gemini is a huge team while Gemma is relatively small.
They can totally do this at a loss with no ulterior motive.
They remind me a bit of HuggingFace, create something great then make money … maybe.
Re: Gemma 4 12B: A unified, encoder-free multimodal model
#123Is this Mac only? Or is that an Ollama issue that it only supports this release of models on Mac? It seems like every tag with the MLX badge is only supported on Mac[0], and that includes all of the tags in this release. [0] https://ollama.com/library/gemma4/tags Edit: MLX being Mac-only is independent of the model being MLX (and therefore Mac) only. The latter is what I am asking about.
Re: Gemma 4 12B: A unified, encoder-free multimodal model
#124A small dense multimodal model with audio support, interesting. Wait, *Excluding Chinese language. This is ... curious. P.S. Where is gemma 4 124b?
Where are the computers we could purchase to run 124b models :’(
Re: Gemma 4 12B: A unified, encoder-free multimodal model
#125Earlier quoted context omitted.
> That's technically encoding Isn't that just projecting the patches into the d_model size vectors that the models takes? >I am assuming that involves of quantization 12B model in 16GB seems very reasonable to me, int8 is top quality for running models.
The guide describes it as projection although there is apparently an extra step: "A factorized coordinate lookup (X and Y matrices) attaches spatial location information directly to the input." 12B at int8 would take up 12G memory, or 75% of the system memory which technically fits within 16GB but the OS will not like that. EDIT: On my 18G memory MacBook Pro, LM Studio reports a "partial GPU offload" for the int8 MLX…
Yeah and it’s pretty memory efficient with only 8 attention layers so at int8 in 16GB ram maybe you still get 64k-128k context.
The part I hate though is that I’d bet none of the performance claims are based on int8.
Why do we care about bf16 benchmarks when no one will be using that with this model.
Re: Gemma 4 12B: A unified, encoder-free multimodal model
#126Earlier quoted context omitted.
They are effectively one and the same on Apple Silicon.
Which most people as a matter of fact don't use. A majority of people with laptop have separate memory pools and the VRAM of them is nowhere near that and even on most gaming laptops you aren't getting 16GB VRAM.
I would say on this forum it wouldn’t be suprising for commentors to be near or above 50% that have access to an M Series Mac…
Re: Gemma 4 12B: A unified, encoder-free multimodal model
#127The big story here is the encoder-free part, which I still don't fully understand. > Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. That's technically encoding, just without using a dedicated model for it like SigLIP? The Developer's Guide elaborates, it's still a 35M layer which I am curious is ro…
I would contend that the actual big story is the gallery app:
https://developers.google.com/edge/gallery
Anyone with a 16GB Mac — that is quite a lot of journalists, surely — can download that, install a model into it, and play.
Surely journalists have to start asking questions at least about OpenAI's consumer revenue projections now.
I am a major, major AI cynic, but I decided to be an informed cynic so I've been playing with local models for agentic work and a bit of CAD-to-image generation. I really quite like the 26B Gemma model — I've been using it to teach myself some fundamental things and learn OpenCode without developing a cloud dependency. It writes fairly good code and it is helping me learn the things I want to learn at a pace that I prefer.
But if this 12B model is even half as close as they say it is, this casts some doubt on the consumer end of the cloud business model, at least in the short term.
(Not clear if this app is using the MTP drafters; I've still not got them working with Gemma myself, though the Qwen 3.6 built-in MTP support is super in LM Studio)
Re: Gemma 4 12B: A unified, encoder-free multimodal model
#128Last time I tried Gemma 4 (26B-A4B) its memory usage would balloon and consume all of my swap until my machine died.
Qwen 3.6 on the other hand barely uses any memory at all for its KV cache.
Re: Gemma 4 12B: A unified, encoder-free multimodal model
#129It's quite interesting to see the quants pour into the HF page. I keep refreshing it and see many new quants every few mins.
Re: Gemma 4 12B: A unified, encoder-free multimodal model
#130What are the use cases for these small models? Is there anyone using models of this scale in their daily life who could share their experience?
I use small models like Gemma to improve transcriptions from ASR models amongst other micro-tasks. I actually built out a fine-tuning whisper pipeline with all local (smaller) models meaning no cloud/big-tech co is able to train/sell my (private) data.
Repo is https://github.com/Rebreda/listenr - mainly geared toward Whisper fine-tuning, AMD hardware and local inference