Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

91–100 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#91

Post by echelon »

gen220 wrote: post #75
zozbot234 wrote: post #68

Earlier quoted context omitted.

A 12B-sized model is a far cry from "frontier inference". That's more like DeepSeek V4 Pro territory which is a 1.6T model. Or for multi-modal models, Kimi 2.6 which is 1T.

at risk of quoting myself... :) > By offering frontier inference closer to cost *and* open-sourcing everything that's sub-frontier It's two prongs! One prong is that their frontier inference pricing is significantly cheaper/closer-to-at-cost as Anthropic's. The subject of this thread is the other prong: offering compelling models that are sub-frontier and self-hostable. Self-hosting models and at-cost frontier models…

Google needs an anti-trust breakup about 10 years ago.

They need one more than ever now.

This is ridiculously anti-competitive.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#92

Post by neosat »

make3 wrote: post #84
mchinen wrote: post #72

Earlier quoted context omitted.

The audio side is even more interesting, as it seems they totally got rid of positional embedding are just doing a single linear transform to match the LLM input dimension and that's it. > Audio: We simplified audio processing even further. We removed the audio encoder entirely and projected the raw audio signal into the same dimensional space as text tokens.

I guarantee you there's positional information one way or another. they just don't mention it because positional embeddings are extremely cheap computationally, not worth mentioning

Agree. Audio has strongly temporal so there is almost certainly some positional encoding one way or another.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#93

Post by joaogui1 »

spott wrote: post #88

Is there a paper on this? I'm curious how they pre-trained it... I feel like it must have had audio/image output that they chopped off. I wonder how hard it would be to add it back on.

I mean Claude is multimodal on input but not output, why couldn't this also be?

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#94

Post by philipkglass »

djyde wrote: post #36

What are the use cases for these small models? Is there anyone using models of this scale in their daily life who could share their experience?

I have vLLM running on a Linux machine in my basement, connected with Tailscale, and I use small models as part of tasks like this:

- Transcribing scanned documents into formatted text

- Captioning/describing images and classifying them for audience suitability (includes anti-spam)

- Matching documents with relevant Wikipedia pages for tagging

I don't use them like frontier models. I break the work down into micro-tasks with one clear goal for each prompt. I write a lot of glue software to make the complete flow work. I was working on all of these tasks before LLMs appeared on the scene. The LLMs have allowed me to replace a lot of complicated code with less code plus a model, while achieving better results.

I use local models for reasons of cost and control. I already had the workstation and GPU. The only running cost is electricity. I have used proprietary models from OpenAI and Google for some of these tasks, but I also encountered churn when the models I built my tools around were retired. I don't worry about that when I have the weights saved locally.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#95

Post by mhitza »

djyde wrote: post #36

What are the use cases for these small models? Is there anyone using models of this scale in their daily life who could share their experience?

In theory, locally you'd use these where lossiness is acceptable for audio transcription and image labeling (as simple examples).

In practice I haven't got around to building something around multimodality since I'm primarily using their text generation capabilities.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#96

Post by utternerd »

vehemenz wrote: post #73
digdugdirk wrote: post #50

I do enjoy the immediate out of touch signaling with the "runs on your 16gb vram laptop" line. Because everyone has a laptop with 16gb vram, or can just pop out and buy a new one, right?

This comment has me a bit confused. Consumers were complaining about the standard 8GB with the early 2020 refresh of MacBook Pros, many OSes ago. Sure, it might be workable for many tasks (as evidenced by the recent sales of the MacBook Neo), but users with a mere 8GB shouldn't have expectations of LLM performance. Even 16GB feels like a stretch.

Unified Memory or VRAM, not just RAM.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#97

Post by Schiendelman »

NekkoDroid wrote: post #82
vehemenz wrote: post #73

Earlier quoted context omitted.

This comment has me a bit confused. Consumers were complaining about the standard 8GB with the early 2020 refresh of MacBook Pros, many OSes ago. Sure, it might be workable for many tasks (as evidenced by the recent sales of the MacBook Neo), but users with a mere 8GB shouldn't have expectations of LLM performance. Even 16GB feels like a stretch.

I think you are mixing up RAM and VRAM.

On a Mac they are the same thing; they're shared. Of course you need some amount for the OS, but if you have an Apple Silicon Mac with 24GB of RAM, you can likely run a 16GB model.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#98

Post by crims0n »

NekkoDroid wrote: post #82
vehemenz wrote: post #73

Earlier quoted context omitted.

This comment has me a bit confused. Consumers were complaining about the standard 8GB with the early 2020 refresh of MacBook Pros, many OSes ago. Sure, it might be workable for many tasks (as evidenced by the recent sales of the MacBook Neo), but users with a mere 8GB shouldn't have expectations of LLM performance. Even 16GB feels like a stretch.

I think you are mixing up RAM and VRAM.

They are effectively one and the same on Apple Silicon.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#99

Post by NekkoDroid »

crims0n wrote: post #98
NekkoDroid wrote: post #82

Earlier quoted context omitted.

I think you are mixing up RAM and VRAM.

They are effectively one and the same on Apple Silicon.

Which most people as a matter of fact don't use. A majority of people with laptop have separate memory pools and the VRAM of them is nowhere near that and even on most gaming laptops you aren't getting 16GB VRAM.
Post reply on HN