Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

291–300 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#291

Post by dgacmu »

I was excited about this until I fed it one of my local test problems: coin identification. I then spent 10 minutes arguing with it that a photo of a 1998 washington quarter was not, in fact, a Morgan Silver Dollar. I mean, I wish it was.

It went into a crash loop on a british columbia 1 dollar coin. This happened with both Q4_1 and Q8. Maybe I'm holding it wrong or it's just really bad for this task.

In contrast, gemma4 gets the british columbia coin right though it also mis-identifies the quarter. gemini 3.1-flash-lite nails them both.

Was getting about 50 t/s output on a 3090 with Q8 which seems ok.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#292

Post by zozbot234 »

SwellJoe wrote: post #285
pseudollm wrote: post #282

Earlier quoted context omitted.

> usefulness of the RTX Spark Not really. There's a reason the announcement didn't include ANY benchmark (!) and didn't mention EXACTLY what is the memory bandwidth. It's going to be dog-slow unusable for large models, as tok/sec is basically bandwidth divided by active weights. Rumoured 300GB/s / 30GB active weights (decent model) = 10 tokens per second, which is really slow

Yep, I have a Strix Halo and while it can run models bigger than Qwen 3.6 27b, it's not usable interactively when you do. ds4 patched for ROCm works, but at such a slow speed, it's not usable for coding agents. The Nvidia boxes have only slightly more memory bandwidth, so I wouldn't expect them to be notably faster. At least not enough to make it useful interactively at that scale.

Why does everyone expect interactivity from local AI? It's not the best use of the hardware, especially not miniPC hardware. Long-term batched inference with larger and more capable models is much more feasible AIUI.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#293

Post by jack_pp »

Aachen wrote: post #283
jack_pp wrote: post #280

Earlier quoted context omitted.

Good point but still, why would Google police this model? If they had a restrictive licence on it do you think it would be worth it for them to enforce it? This way they at least buy some good will and mindshare

That makes sense to me. Guess one might say the same for game icons and other such files that lay around in disks, but yeah maybe it's as simple as that

Not quite the same, understandably Blizzard cares a lot about their IP because otherwise private servers leech their users. Maybe a small game designer cares a lot about the small game they made or whatever since that's all they have. A four trillion market cap company can afford to be "charitable".. where it costs them nothing and might cost them more to enforce their rights.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#294

Post by doubled112 »

iso1631 wrote: post #281
0xbadcafebee wrote: post #206

Earlier quoted context omitted.

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

I find ram crazy. My thinkpad has 32G of ram, it's a t470 that's nearly a decade old Why do people with modern laptops have such little amounts of ram?

My job still issues 16GB laptops as standard. You need a business reason to get more. This has been going on since before the price hikes.

I’m a system administrator and I can do my job with no issues at 16GB. Most days 8GB would likely be enough, since I’m just using and abusing other systems anyway.

Java devs at my last job were still running 16GB in 2020. Admittedly that was a while ago. Still not a decade.

Close some Chrome tabs?

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#295

Post by willy_k »

iso1631 wrote: post #281
0xbadcafebee wrote: post #206

Earlier quoted context omitted.

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

I find ram crazy. My thinkpad has 32G of ram, it's a t470 that's nearly a decade old Why do people with modern laptops have such little amounts of ram?

The ram that’s important for LLMs is gpu-accessible memory, meaning either systems with unified ram or VRAM, the latter of which is tied to the caliber of GPU one has.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#296

Post by SwellJoe »

zozbot234 wrote: post #292
SwellJoe wrote: post #285

Earlier quoted context omitted.

Yep, I have a Strix Halo and while it can run models bigger than Qwen 3.6 27b, it's not usable interactively when you do. ds4 patched for ROCm works, but at such a slow speed, it's not usable for coding agents. The Nvidia boxes have only slightly more memory bandwidth, so I wouldn't expect them to be notably faster. At least not enough to make it useful interactively at that scale.

Why does everyone expect interactivity from local AI? It's not the best use of the hardware, especially not miniPC hardware. Long-term batched inference with larger and more capable models is much more feasible AIUI.

Even batched it's uncomfortably slow. I started to benchmark ds4 with my security vulnerability benchmark (after Qwen 3.6 dense and MoE and a bunch of cloud models), but it was going to tie up the Strix Halo for more than a day, so I decided not to run it as it would prevent me from doing other stuff with it during that time.

Even batched usage needs to be fast enough to deliver results in a reasonable time. Overnight runs are useful, 24 hour runs are...less so.

Anyway, most of the time people are talking about interactive use, and there's currently an upper bound on how large a model can be for local hosting on a reasonable budget (i.e. not a crazy amount more expensive than what a high end developer desktop or laptop costs). The sweet spot is probably currently the big Qwen 3.6 or Gemma 4 models, which are in the ~60GB range for 8-bit quantization plus a large context.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#297

Post by kordlessagain »

scirob wrote: post #182

Quickly deployed it to check some benchmarks relevant for German language. These are results for CohereLabs/include-base-44 german only : Gemma 4 12B %61.9 Gemma 4 26B (a4b MoE) 0.647 Qwen 3 14B 0.621 Gemma 4 12B 0.618 Ministral 14B 2512 0.604 Gemma 3 12B 0.547 The quwen 3 14B vs Gemma 4 12B difference is within random variance they same in some repeat runs they actually got the exact same score. Next step up Gemma 4…

I just ran a short tool use test and it's doing pretty well.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#298

Post by kordlessagain »

lxgr wrote: post #201
lxgr wrote: post #60

Am I missing something or are the Ollama versions of this ( https://ollama.com/library/gemma4/tags ) text-only for now?

To anybody else wondering: Seems like the models supporting image input are just starting to show up. https://ollama.com/library/gemma4:12b-mlx now shows as supporting it, but curiously the overview on https://ollama.com/library/gemma4/tags still lists it as text only. Cache invalidation remains difficult :)

Yup, the new version of Ollama dropped. Time to update.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#299

Post by sureglymop »

dofm wrote: post #127
minimaxir wrote: post #2

The big story here is the encoder-free part, which I still don't fully understand. > Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. That's technically encoding, just without using a dedicated model for it like SigLIP? The Developer's Guide elaborates, it's still a 35M layer which I am curious is ro…

I would contend that the actual big story is the gallery app: https://developers.google.com/edge/gallery Anyone with a 16GB Mac — that is quite a lot of journalists, surely — can download that, install a model into it, and play. Surely journalists have to start asking questions at least about OpenAI's consumer revenue projections now. I am a major, major AI cynic, but I decided to be an informed cynic so I've been pl…

Is the story that it's now also available outside of android? I've had this app on my phone for I believe about a year.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#300

Post by ai_fry_ur_brain »

airstrike wrote: post #254
ai_fry_ur_brain wrote: post #246

Earlier quoted context omitted.

Why would I want an AI receptionist. A human receptionist is about 1000x more careful, caring and intentional. They are charging $15.00 an hour for an llm powered assistant. Like wtf, how do these people think that's a valid business model. This will 1000% annoy every customer that uses it. I hate this timeline so much.

No, this is a phone service. They charge $0.25 per minute on the phone on a call that would otherwise not connect. Can you call a receptionist at 10pm and book an appointment? Or ask for directions? What if it's 10am and she's already on the line with someone else and you just want to ask if there's parking?

Please tell me what 0.25c x 60 is.

Yes, they're called after hours answering services and they're exponentially better because I get to talk to a human.

If my doctors office replaced a receptionist with this I would switch and leave bad reviews across every platform possible.

Ive already switched doctors once because they used an LLM transcription service during my appoitment that influenced the doctors recommendations for care. Sorry technology does not belong everywhere.

AI produces low quality work and will turn your business to shit.

Post reply on HN