Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

191–200 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#191

Post by Patrick_Devine »

embedding-shape wrote: post #40
Zambyte wrote: post #32

Is this Mac only? Or is that an Ollama issue that it only supports this release of models on Mac? It seems like every tag with the MLX badge is only supported on Mac[0], and that includes all of the tags in this release. [0] https://ollama.com/library/gemma4/tags Edit: MLX being Mac-only is independent of the model being MLX (and therefore Mac) only. The latter is what I am asking about.

MLX is quite literally macOS-specific technology, for other platforms you want non-MLX. I was sure "MLX" stood for "Metal-something-something" but can't find any reference to that somehow, anywho, "Metal" is hardware-accelerated graphics on Apple platforms FWIW. Edit: about the actual release on Ollama, if you're on non-Apple hardware you probably want the NVFP4 variant ("gemma4:12b-nvfp4") which was uploaded 45 minu…

I realize this is a little confusing; we're working w/ the MLX team to bring MLX to other platforms, but we're not quite there yet. The `gemma4:12b-nvfp4` model is specifically for the MLX engine.

For the GGUF 4bit variant (i.e. non-macs) you'll need `gemma4:12b-it-q4_K_M` which I just pushed. You'll also need to upgrade to version 0.30.4 which we're just about to release (it's in prerelease and we're running through our last regression tests).

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#192

Post by dofm »

ch_sm wrote: post #189
dofm wrote: post #175

Earlier quoted context omitted.

Could you outline how you are running the MTP drafters? I've tried LM Studio but no dice there. I'm probably missing something but I think llama.cpp and Ollama can't do it yet either?

can‘t speak to compatibility with this new model, but oMLX supports MTP drafters very well.

Thank you, I will test that.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#193

Post by dofm »

Patrick_Devine wrote: post #188
dofm wrote: post #175

Earlier quoted context omitted.

Could you outline how you are running the MTP drafters? I've tried LM Studio but no dice there. I'm probably missing something but I think llama.cpp and Ollama can't do it yet either?

I haven't yet pushed the MTP enabled gemma4 12b model for Ollama because in my testing I wasn't getting a performance bump. The other gemma4 MTP models should work OK right now, but there are some fixes we're just about to push. This is specifically for the MLX backend.

Thanks for your reply. I will go back and look at Ollama again.

So much to learn but this news has really vindicated my decision to direct my limited span of concentration and focus to learning how to use open weights models and opencode.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#194

Post by mugivarra69 »

ethanpil wrote: post #13

What's Google's business case for releasing open models? Don't get me wrong, I am grateful and appreciative of these releases. I'm trying to understand how it fits into their bigger picture as a for profit company? Are they not helping competitors build on the novel technology they have developed? Is it simply goodwill and/or marketing? Or am I missing something strategic?

[dead]

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#195

Post by properbrew »

SeriousM wrote: post #131
properbrew wrote: post #80

Earlier quoted context omitted.

I think small models have a very good niche for specific tasks. I utilise a fine tuned Phi-4 model (smaller than this one) that fits in about 3.5gb of RAM (not vram) for the document processing side of things for the desktop app I develop (a bit of a shameless plug - whistle-enterprise.com). If you have a very specific idea for local model use you can find a way to make it work very well, you don't even need to have…

Thank you for sharing your usecase! I like your product very much! Could you talk a bit how you did the finetuning? Did you use unsloth or any other tool and how went the verification to proof the outcome?

Thank you!

Yea absolutely, but man, where to even start, it is very specific.

Fundementally I didn't use any wrappers like unsloth or axolotl, although I have used the latter before a year or two back and it was good, but I needed something very very custom. I also wanted the whole fine tuning pipeline to exported OpenVino model to be seamless.

I heavily leaned on codex, claude and some manual sleuthing around the internet to understand what I needed. I'd played about with QLoRA finetuning with axolotl before and felt most comfortable with that. So I needed to keep everything as stripped down as possible and figured I can just utilise the 3 main huggingface libraries (transformers, peft and datasets) and also bitsandbytes (as suggested by claude to quantize the model to keep this working on my GPU) along with some custom scripts generated by claude/codex (each cross referencing each other) that will do the different stages of the training run.

The next part was the data. Obviously didn't have access to thousands of meetings and associated output documents but I did have a 3090ti sitting there and a codex subscription. So I set about working out what format I needed the data in (many thanks again, to claude/codex) and started generating hundreds of different transcripts, different amounts of speakers, content, tones, subjects, spelling mistakes - like all the different things you could think a meeting would have. Then it's a case of actually generating a good meeting document off the back of the transcripts and creating the "gold standard" that we'd use.

I'm going to gloss over a lot here as I'd rather not detail it as it relates to some propriatary stuff that I had to work through, but you basically pair the transcripts together and run the training.

At the verification stage, there was pretty much 3 things:

1. "just" do some regex string matching to see if there's any of the source transcript key facts in the output to ensure fact preservation. Same with owner fabrication (who said what), I don't want something attributed to someone when it wasn't them that said it and then finally markdown validation.

2. Using codex/claude to validate the transcript and output from the model - I used the latest frontier models, probably overkill for my task, but they were good at the job

3. Finally me going through some actual recordings of myself, groups, meetings and manually verifiying the output

So a fair bit of work, and for context I'm on version 10 now, so it's been a journey!

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#196

Post by selectodude »

altruios wrote: post #104
alberto467 wrote: post #49

Earlier quoted context omitted.

Not at all. Getting really pedantic, tokenization is also a form of encoding, so it doesn't matter the modality you're using, you'll end up doing some type of encoding in some way.

Tokens are such a strange base unit. Couldn't we do something that naturally conforms better to reality than such choppy units that cause all sorts of artifacts? making everything 'language based' prevents true multi-modality. Thinking isn't done in language. Thinking outputs language, but its far more like multiple waves of data coalescing into an 'idea', internal... subjectively (n=1) at least. I think wave/signal…

Not to be too snarky but there’s a few trillion dollars and some of the brightest minds of our generation working on this. I’m sure there’s a reason why they’ve settled for or are stuck on tokenization.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#197

Post by _puk »

verdverm wrote: post #152
mchusma wrote: post #143

Earlier quoted context omitted.

Gemma 4 31b outperformed Gemini 3.1 Flash-Lite in our app benchmarks (agentic tool use via api in our application as a part of various workflows). But google won't let you pay to use Gemma models, you have to go elsewhere, I think this may be because it would cannabilize Flash-lite.

You can actually get the gemma-4 models on a per-token API basis, you just have to click some extra buttons (in GCP). Not the same for other open weight models. For those they make you run your own hardware. Use OpenCode Go instead: https://opencode.ai/go

That doesn't have the Gemma models by the looks

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#199

Post by McGlockenshire »

senko wrote: post #120

I ran the Q4 quant (used with llama.cpp) though my "minesweeper" vibe-coding benchmark: https://senko.net/vibecode-bench/2026/minesweeper-gamma-4-12... The result is decent, but it had a few bizzare/trivial syntax errors I had to fix manually: it would do an extra closing bracket or paren a few times, and wanted to separate function definitions with comma. Not sure what that was about, but otherwise the output run ju…

> my consumer-grade card with 12G of VRAM and got 5t/s for output

Thank you for giving me hope!

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#200

Post by refulgentis »

altruios wrote: post #104
alberto467 wrote: post #49

Earlier quoted context omitted.

Not at all. Getting really pedantic, tokenization is also a form of encoding, so it doesn't matter the modality you're using, you'll end up doing some type of encoding in some way.

Tokens are such a strange base unit. Couldn't we do something that naturally conforms better to reality than such choppy units that cause all sorts of artifacts? making everything 'language based' prevents true multi-modality. Thinking isn't done in language. Thinking outputs language, but its far more like multiple waves of data coalescing into an 'idea', internal... subjectively (n=1) at least. I think wave/signal…

This sounds like when crystal people talk quantum physics.
Post reply on HN