Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

341–350 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#341

Post by amelius »

georgehm wrote: post #53
minimaxir wrote: post #2

The big story here is the encoder-free part, which I still don't fully understand. > Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. That's technically encoding, just without using a dedicated model for it like SigLIP? The Developer's Guide elaborates, it's still a 35M layer which I am curious is ro…

Embedded within that developer page is a good explainer of the encoder free architecture . https://newsletter.maartengrootendorst.com/p/a-visual-guide-...

I skimmed it, but I still wonder why (1) we still need a tokenizer for text, and (2) why the other modalities (audio/video) don't need one.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#342

Post by SturgeonsLaw »

iso1631 wrote: post #281
0xbadcafebee wrote: post #206

Earlier quoted context omitted.

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

I find ram crazy. My thinkpad has 32G of ram, it's a t470 that's nearly a decade old Why do people with modern laptops have such little amounts of ram?

Unified memory is soldered to the motherboard and needs to be ordered with the new laptop, for prices that are well above what the equivalent amount of SODIMM would cost.

Fine if work's paying, but for personal devices (that might have been purchased before local models got good), people have what they have.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#344

Post by dotancohen »

akmarinov wrote: post #324
dotancohen wrote: post #273

Earlier quoted context omitted.

> For 16GB laptops, Qwen 3.5 9B is the undisputed champ. You seem like the guy to ask. For a laptop with 12GB VRAM (RTX 5070) and 32 GB system RAM, what is a good multilingual (English, Hebrew, Greek) model for conversing with personal notes in Org mode format? I don't care how long updating the model or rag takes, and even inference can be reasonably slow, but the results of the query as they relate to my personal n…

Joins us over on Reddit at r/LocalLlaMA to get 10 different opinions on that

I read there regularly. I find little value there between the memes. I was hoping to ask a knowledgeable person here.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#345

Post by yorwba »

altruios wrote: post #264
TeMPOraL wrote: post #241

Earlier quoted context omitted.

> making everything 'language based' prevents true multi-modality. Thinking isn't done in language. Thinking outputs language Your problem isn't with tokens, but with "language". Tokens have little to do with language, other than usually being consumed in sequence, but that's true of anything that has to span over time. Thinking of tokens as letters or subwords is mistaking the general with the specific. We may have…

Can you elaborate more on what a token looks like as a pixel patch/sound/general signal as it currently is (in this model)? My understanding of pixel representation is: slice a grid in an image, each square slice gets projected into a number array of x long (not sure how long x is, or if it's variable), which then gets projected down to a token representing that space (3-4 long as alpha-numeric) and AGAIN gets passed…

> number array of x long (not sure how long x is, or if it's variable), which then gets projected down to a token representing that space (3-4 long as alpha-numeric)

There is no such projection step. The array of x numbers is the token. For text, there is a one-to-one correspondence between the textual representation of a token, its index in the vocabulary of the model, and the array of x numbers that is fed into the linear algebra of the model, so people often equivocate between them; but for images or sound, there is no discrete vocabulary and no textual representation, only the array of x numbers.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#346

Post by ddorian43 »

asim wrote: post #137

We are now entering the closed loop game. Google doesn't need anyone else to accelerate their models. This is their bread and butter. I'm both shocked but also not surprised that they continue to develop such efficiencies. Honestly it's like silicon and CPU architecture advancement. We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x…

> people will live to 125 quite steadily

Only after the current generation(s) of doctor(s) dies. And only if you make this in pill-form. Otherwise people will be people and won't even go to the gym.

It might be also the reverse, they develop a powerful+personalized drug that brings heaven on earth to your neurons (first time heroin experience + sexual gratification + childhood fulfillment + extremely addicting etc etc etc).

-----

Now that I think of it I'm gonna go with the latter.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#347

Post by embedding-shape »

MoonWalk wrote: post #277
xenophonf wrote: post #268

Earlier quoted context omitted.

It took me way too long to realize you were referring to r/localllama.

Why the obfuscation in the first place?

Just a bit of flair. Also, bunch of people have "keyword watchers" setup for various terms, so when you mention certain things on HN, reddit and elsewhere, you get commentators who enter the conversation not because the context or larger conversation, but because the single term/thing they care deeply about was mentioned, and it just gets very boring to read the whole attackers/defenders comments over and over again. But ultimately I just did it like that because it was more fun to write it like that.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#348

Post by hparadiz »

hypfer wrote: post #338
emmelaich wrote: post #311

Earlier quoted context omitted.

You may like https://www.llmfit.org/ (not recommendation, I've not used it .. yet)

Just tried it and honestly it's a terrible experience lacking any sort of intent or reason. Which is unsurprising in the AI space. You get a wall of text showing you various random fine-tuned models by random people, and that is basically it. Actual sane default requirements like "just give me the normal AI labs", "please filter for dense only" and "I want this exact context size at this quant" are not part of the to…

We need benchmarks by engine, cli switch sets, and device with filters by cpu, gpu, and type. And if someone could please aggregate that in a way where people can upload results and just automatically see the best of any model for their device that would be a killer app.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#349

Post by alfiedotwtf »

dotancohen wrote: post #344
akmarinov wrote: post #324

Earlier quoted context omitted.

Joins us over on Reddit at r/LocalLlaMA to get 10 different opinions on that

I read there regularly. I find little value there between the memes. I was hoping to ask a knowledgeable person here.

/r/localllama for a while now seems to prefer Gemma 4 E4B for creative writing (especially the uncensored GGUFs).

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#350

Post by alfiedotwtf »

hparadiz wrote: post #348
hypfer wrote: post #338

Earlier quoted context omitted.

Just tried it and honestly it's a terrible experience lacking any sort of intent or reason. Which is unsurprising in the AI space. You get a wall of text showing you various random fine-tuned models by random people, and that is basically it. Actual sane default requirements like "just give me the normal AI labs", "please filter for dense only" and "I want this exact context size at this quant" are not part of the to…

We need benchmarks by engine, cli switch sets, and device with filters by cpu, gpu, and type. And if someone could please aggregate that in a way where people can upload results and just automatically see the best of any model for their device that would be a killer app.

I've wanted to vibe code a tuning app, that pumps data through your CPU-GPU-RAM to try and determine the best parameters for each model, but I think it's just too much work compared to manually running by hand a one-liner and changing things here and there.
Post reply on HN