Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google releases Gemma 4 open models

deepmind.google

Post reply on HN
Print view View original

111–120 of 507 posts

Re: Google releases Gemma 4 open models

#111

Post by mhitza »

wg0 wrote: post #18

Google might not have the best coding models (yet) but they seem to have the most intelligent and knowledgeable models of all especially Gemini 3.1 Pro is something. One more thing about Google is that they have everything that others do not: 1. Huge data, audio, video, geospatial 2. Tons of expertise. Attention all you need was born there. 3. Libraries that they wrote. 4. Their own data centers and cloud. 4. Most of…

At the start of last year Gemma2 made the fewest mistakes when I was trying out self-hosted LLMs for language translation. And at the time it had a non open source license.

Really eager to test this version with all the extra capabilities provided.

Re: Google releases Gemma 4 open models

#112

Post by logicchains »

whimblepop wrote: post #88
wg0 wrote: post #18

Google might not have the best coding models (yet) but they seem to have the most intelligent and knowledgeable models of all especially Gemini 3.1 Pro is something. One more thing about Google is that they have everything that others do not: 1. Huge data, audio, video, geospatial 2. Tons of expertise. Attention all you need was born there. 3. Libraries that they wrote. 4. Their own data centers and cloud. 4. Most of…

I recently canceled my Google One subscription because getting accurate answers out of Gemini for chat is basically impossible afaict. Whether I enable thinking makes no difference: Gemini always answers me super quickly, rarely actually looks something up, and lies to me. It has a really bad unchecked hallucination problem because it prioritizes speed over accuracy and (astonishingly, to me) is way more hesitant to…

Recently I had a pretty basic question about whether there was a Factorio mod for something so decided to ask it to Gemini, it hallucinated not one but two sadly non-existing mods. Even Grok is better at search.

Re: Google releases Gemma 4 open models

#113

Post by virgildotcodes »

Downloaded through LM Studio on an M1 Max 32GB, 26B A4B Q4_K_M

First message:

https://i.postimg.cc/yNZzmGMM/Screenshot-2026-04-03-at-12-44...

Not sure if I'm doing something wrong?

This more or less reflects my experience with most local models over the last couple years (although admittedly most aren't anywhere near this bad). People keep saying they're useful and yet I can't get them to be consistently useful at all.

Re: Google releases Gemma 4 open models

#114

Post by nabakin »

moffkalast wrote: post #94
nabakin wrote: post #38

Earlier quoted context omitted.

Public benchmarks can be trivially faked. Lmarena is a bit harder to fake and is human-evaluated. I agree it's misleading for them to hyper-focus on one metric, but public benchmarks are far from the only thing that matters. I place more weight on Lmarena scores and private benchmarks.

Lm arena is so easy to game that it's ceased to be a relevant metric over a year ago. People are not usable validators beyond "yeah that looks good to me", nobody checks if the facts are correct or not.

It's easy to game and human evaluation data has its trade-offs, but it's way easier to fake public benchmark results. I wish we had a source of high quality private benchmark results across a vast number of models like Lmarena. Having high quality human evaluation data would be a plus too.

Re: Google releases Gemma 4 open models

#115

Post by danielhanchen »

zaat wrote: post #95
danielhanchen wrote: post #2

Thinking / reasoning + multimodal + tool calling. We made some quants at https://huggingface.co/collections/unsloth/gemma-4 for folks to run them - they work really well! Guide for those interested: https://unsloth.ai/docs/models/gemma-4 Also note to use temperature = 1.0, top_p = 0.95, top_k = 64 and the EOS is " ". " thought\n" is also used for the thinking trace!

Thank you for your work. You have an answer on your page regarding "Should I pick 26B-A4B or 31B?", but can you please clarify if, assuming 24GB vRAM, I should pick a full precision smaller model or 4 bit larger model?

Thank you!

I presume 24B is somewhat faster since it's only 4B activated - 31B is quite a large dense model so more accurate!

Re: Google releases Gemma 4 open models

#116

Post by jug »

moffkalast wrote: post #94
nabakin wrote: post #38

Earlier quoted context omitted.

Public benchmarks can be trivially faked. Lmarena is a bit harder to fake and is human-evaluated. I agree it's misleading for them to hyper-focus on one metric, but public benchmarks are far from the only thing that matters. I place more weight on Lmarena scores and private benchmarks.

Lm arena is so easy to game that it's ceased to be a relevant metric over a year ago. People are not usable validators beyond "yeah that looks good to me", nobody checks if the facts are correct or not.

I agree; LMArena died for me with the Llama 4 debacle. And not only the gamed scores, but seeing with shock and horror the answers people found good. It does test something though: the general "vibe" and how human/friendly and knowledgeable it _seems_ to be.

Re: Google releases Gemma 4 open models

#117

Post by logicallee »

canyon289 wrote: post #51

Hi all! I work on the Gemma team, one of many as this one was a bigger effort given it was a mainline release. Happy to answer whatever questions I can

Do any of you use this as a replacement for Claude Code? For example, you might use it with openclaw. I have a 24 GB integrated RAM Mac Mini M4 I currently run Claude Code on, do you think I can replace it with OpenClaw and one of these models?

Re: Google releases Gemma 4 open models

#118

Post by bachmeier »

scrlk wrote: post #15

Comparison of Gemma 4 vs. Qwen 3.5 benchmarks, consolidated from their respective Hugging Face model cards: | Model | MMLUP | GPQA | LCB | ELO | TAU2 | MMMLU | HLE-n | HLE-t | |----------------|-------|-------|-------|------|-------|-------|-------|-------| | G4 31B | 85.2% | 84.3% | 80.0% | 2150 | 76.9% | 88.4% | 19.5% | 26.5% | | G4 26B A4B | 82.6% | 82.3% | 77.1% | 1718 | 68.2% | 86.3% | 8.7% | 17.2% | | G4 E4B |…

So is there something I can take from that table if I have a 24 GB video card? I'm honestly not sure how to use those numbers.

Re: Google releases Gemma 4 open models

#119

Post by solarkraft »

virgildotcodes wrote: post #113

Downloaded through LM Studio on an M1 Max 32GB, 26B A4B Q4_K_M First message: https://i.postimg.cc/yNZzmGMM/Screenshot-2026-04-03-at-12-44... Not sure if I'm doing something wrong? This more or less reflects my experience with most local models over the last couple years (although admittedly most aren't anywhere near this bad). People keep saying they're useful and yet I can't get them to be consistently useful at al…

Wow, just like its larger brother!

I had a similarly bad experience running Qwen 3.5 35b a3b directly through llama.cpp. It would massively overthink every request. Somehow in OpenCode it just worked.

I think it comes down to temperature and such (see daniel‘s post), but I haven’t messed with it enough to be sure.

Re: Google releases Gemma 4 open models

#120

Post by jimbokun »

echelon wrote: post #17
rvz wrote: post #11

Open weight models once again marching on and slowly being a viable alternative to the larger ones. We are at least 1 year and at most 2 years until they surpass closed models for everyday tasks that can be done locally to save spending on tokens.

> We are at least 1 year and at most 2 years until they surpass closed models for everyday tasks that can be done locally to save spending on tokens. Until they pass what closed models today can do. By that time, closed models will be 4 years ahead. Google would not be giving this away if they believed local open models could win. Google is doing this to slow down Anthropic, OpenAI, and the Chinese, knowing that in t…

But at that point, won’t there be very few tasks left where the average user can discern the difference in quality for most tasks?
Post reply on HN