Gemma 4 12B: A unified, encoder-free multimodal model • Hacker News Board

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#321

Post by gnabgib » Thu, Jun 04, 2026, 4:49 AM UTC

ai_fry_ur_brain wrote: post #300

airstrike wrote: post #254

Earlier quoted context omitted.

No, this is a phone service. They charge $0.25 per minute on the phone on a call that would otherwise not connect. Can you call a receptionist at 10pm and book an appointment? Or ask for directions? What if it's 10am and she's already on the line with someone else and you just want to ask if there's parking?

Please tell me what 0.25c x 60 is. Yes, they're called after hours answering services and they're exponentially better because I get to talk to a human. If my doctors office replaced a receptionist with this I would switch and leave bad reviews across every platform possible. Ive already switched doctors once because they used an LLM transcription service during my appoitment that influenced the doctors recommendatio…

Are you, perhaps, missing that $0.25/minute is only minutes on call? An agent not answering the phone for an hour is $0 (not $15).. for after-hours calls (rare) this is a meagre rate, compared to pay-per-hour (no matter the call volume) answering services.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#322

Post by madduci » Thu, Jun 04, 2026, 5:08 AM UTC

KiwiJohnno wrote: post #320

madduci wrote: post #140

Earlier quoted context omitted.

VRAM, not RAM. I wish it was light enough for iGPUs too

I ran the 26B model on my i5 which has no discrete graphics card. It ran about 7 tokens/sec, and appreared to be a very capable model.

Interesting, thanks for the feedback!

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#323

Post by tacomagick » Thu, Jun 04, 2026, 5:09 AM UTC

dotancohen wrote: post #273

0xbadcafebee wrote: post #206

Earlier quoted context omitted.

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

> For 16GB laptops, Qwen 3.5 9B is the undisputed champ. You seem like the guy to ask. For a laptop with 12GB VRAM (RTX 5070) and 32 GB system RAM, what is a good multilingual (English, Hebrew, Greek) model for conversing with personal notes in Org mode format? I don't care how long updating the model or rag takes, and even inference can be reasonably slow, but the results of the query as they relate to my personal n…

Gemma 4 26A4B

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#324

Post by akmarinov » Thu, Jun 04, 2026, 5:14 AM UTC

dotancohen wrote: post #273

0xbadcafebee wrote: post #206

Earlier quoted context omitted.

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

> For 16GB laptops, Qwen 3.5 9B is the undisputed champ. You seem like the guy to ask. For a laptop with 12GB VRAM (RTX 5070) and 32 GB system RAM, what is a good multilingual (English, Hebrew, Greek) model for conversing with personal notes in Org mode format? I don't care how long updating the model or rag takes, and even inference can be reasonably slow, but the results of the query as they relate to my personal n…

Joins us over on Reddit at r/LocalLlaMA to get 10 different opinions on that

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#325

Post by zkmon » Thu, Jun 04, 2026, 5:22 AM UTC

mdp2021 wrote: post #239

zkmon wrote: post #101

I'm waiting for FP8 quant, preferably from Google.

If you accept the "ggml-org": https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/mai... https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/blob/mai...

Do they run well on vLLM?

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#326

Post by dirkg » Thu, Jun 04, 2026, 5:25 AM UTC

0xbadcafebee wrote: post #206

senko wrote: post #120

I ran the Q4 quant (used with llama.cpp) though my "minesweeper" vibe-coding benchmark: https://senko.net/vibecode-bench/2026/minesweeper-gamma-4-12... The result is decent, but it had a few bizzare/trivial syntax errors I had to fix manually: it would do an extra closing bracket or paren a few times, and wanted to separate function definitions with comma. Not sure what that was about, but otherwise the output run ju…

It was almost certainly not trained for coding, as it's got both audio and vision input, is only 12B, and nowhere in the announcement is coding mentioned. It will likely not have good performance on coding in general, compared to other small models like Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nvidia Nemotron 3 Nano 30B-A3B, gpt-oss-20b. For 16GB laptops, Qwen 3.5 9B is the undisputed champ. Gemma 4 31B is the top dog at s…

> For 16GB laptops, Qwen 3.5 9B is the undisputed champ.

you can run qwen 3.6 35BA3B on a 12-16GB vram gpu and ot works pretty well.

https://www.youtube.com/watch?v=8F_5pdcD3HY&t=1s

even the 27B in some quants can fit.

https://www.reddit.com/r/LocalLLaMA/comments/1tkmgwj/qwen27b...

qwen IMO is far better for coding, esp agentic coding when combined with something like Pi, it comes probably close enough to Sonnet for a lot of use cases.

Gemma family is better for almost all other tasks you'd use a local llm for.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#327

Post by silversmith » Thu, Jun 04, 2026, 5:26 AM UTC

sourcecodeplz wrote: post #309

dotancohen wrote: post #273

Earlier quoted context omitted.

> For 16GB laptops, Qwen 3.5 9B is the undisputed champ. You seem like the guy to ask. For a laptop with 12GB VRAM (RTX 5070) and 32 GB system RAM, what is a good multilingual (English, Hebrew, Greek) model for conversing with personal notes in Org mode format? I don't care how long updating the model or rag takes, and even inference can be reasonably slow, but the results of the query as they relate to my personal n…

Any Gemma 4 model, they are great at translations, multilingual

For the biggest languages, Spanish, French, maybe.

For smaller ones like my native Latvian, the output could be confused for good translation from across the room, the words do look like Latvian words. But the quality is Google translate circa 20 years ago, tops.

It could probably do a decent enough translation to English, if all you need is to get the gist of text. But for smaller European language outputs, nothing comes close to Gemini.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#328

Post by fittingopposite » Thu, Jun 04, 2026, 5:27 AM UTC

robgough wrote: post #81

djyde wrote: post #36

What are the use cases for these small models? Is there anyone using models of this scale in their daily life who could share their experience?

I've got a home-built dictation app that uses a local model to clear up the text and fix grammar. It was super easy to build. I’m extending it to capture meeting notes and summarise too. All on-device. I saw a little app the other day, I think someone posted on here, that looks at your screenshot and renames the file based off the contents of the file. There's tons of little examples like that. For a lot of use cases…

That's a great user case. Am sorry using parakeet but sometimes it garbles up things. Can you open source it?

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#329

Post by dirkg » Thu, Jun 04, 2026, 5:38 AM UTC

pseudosavant wrote: post #259

senko wrote: post #120

I ran the Q4 quant (used with llama.cpp) though my "minesweeper" vibe-coding benchmark: https://senko.net/vibecode-bench/2026/minesweeper-gamma-4-12... The result is decent, but it had a few bizzare/trivial syntax errors I had to fix manually: it would do an extra closing bracket or paren a few times, and wanted to separate function definitions with comma. Not sure what that was about, but otherwise the output run ju…

Models this small and this capable bode really well for the usefulness of a PC like the RTX Spark that Nvidia/Microsoft announced this week. 128GB of unified memory will likely be more than sufficient for effective local agentic coding, even if SOTA cloud models will still be even better. Up until this point, I've found the cost/value to unequivocally favor using a cloud subscription, but I would be lying if I didn't…

The RTX/DGX Spark, Mac Ultras with 128GB unified ram are all ~$5k. Its still an expensive toy for rich people, it might as well be an H100 for 99.9% of the population (not devs with high paying jobs, of course).

the value of local models is allowing normal people to access AI without needing to subscribe to cloud services. this is esp imp for the rest of the world where even a 12GB gpu is extremely expensive.

there is no real viable local option that will come even close to Sonnet/Gemini Flash or the cheaper chinese models. Even if your pc costs <$2k you are never going to recoup the hw costs, and the results will be far worse.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#330

Post by milch » Thu, Jun 04, 2026, 5:44 AM UTC

hedgehog wrote: post #305

SwellJoe wrote: post #296

Earlier quoted context omitted.

Even batched it's uncomfortably slow. I started to benchmark ds4 with my security vulnerability benchmark (after Qwen 3.6 dense and MoE and a bunch of cloud models), but it was going to tie up the Strix Halo for more than a day, so I decided not to run it as it would prevent me from doing other stuff with it during that time. Even batched usage needs to be fast enough to deliver results in a reasonable time. Overnigh…

The 6-bit versions + 8-bit KV cache seems to save a good bit of memory without a significant loss of quality. The Qwen 35B is pretty fast in my testing, but MiniMax M2.7 230B is in some ways faster (way fewer tokens to arrive at an answer) even though it is much larger.

What hardware do you need to run MiniMax M2.7 230B locally?