Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google releases Gemma 4 open models

deepmind.google

Post reply on HN
Print view View original

461–470 of 507 posts

Re: Google releases Gemma 4 open models

#461

Post by mikewarot »

I updated Ollama (again) and changed my windows swap file settings to use up to 200 Gb of C: (an SSD). On the largest model (gemma4:31b), I seem to be getting about 5 tokens per second. This is amazing to me, because I'm using a $100 computer, without any fancy GPU. I love watching it "think".

Consider this is thousands of times faster than any written conversations in the past. Those involved pieces of paper being transported, read, considered, replies written, then transported back.

If it'll write code that doesn't completely suck, I think even this is good enough. What do you consider the lowest acceptable rate of generating tokens/second?

Re: Google releases Gemma 4 open models

#462

Post by kristopolous »

dave7 wrote: post #454
kristopolous wrote: post #428

Earlier quoted context omitted.

This is actually a perfect use case of my llcat ... Using Unix philosophy for the llm era https://github.com/day50-dev/llcat I've got an interesting hack brewing for extremely hassle free tool orchestration - basically think along the lines of .bash_profile level simplicity... Maybe I'll get that out tomorrow

llcat looks like it could be very handy, thank you!

Thanks. I really want to create great quality software.

so File Bugs if you find them. Please!

Re: Google releases Gemma 4 open models

#463

Post by mudkipdev »

mikewarot wrote: post #461

I updated Ollama (again) and changed my windows swap file settings to use up to 200 Gb of C: (an SSD). On the largest model (gemma4:31b), I seem to be getting about 5 tokens per second. This is amazing to me, because I'm using a $100 computer, without any fancy GPU. I love watching it "think". Consider this is thousands of times faster than any written conversations in the past. Those involved pieces of paper being t…

Under 15 is too slow for conversation personally. I guess 5 tokens per second is nice if you're one of the people who likes letting coding agents run overnight

Re: Google releases Gemma 4 open models

#466

Post by bwannasek »

Using Gemma 4 with OpenCode was more challenging than expected due to some active bugs in ollama related to reasoning and streaming - I did a quick writeup in how I used llama.cpp instead of ollama and how to set it up to support multi-turn tool calls properly in case this is helpful to others: https://bernhardwannasek.com/using-gemma-4-for-agentic-codin...

Re: Google releases Gemma 4 open models

#467

Post by ronb1964 »

I have Ollama installed on my Linux desktop with Alpaca as the frontend, but honestly I haven't done much with it beyond poking around. I also built a local speech-to-text app using Claude Code that runs Whisper offline, so I'm clearly drawn to the idea of keeping AI on-device. I'm curious whether Gemma 4 would be a noticeable step up for someone just using a local model for everyday tasks...writing, Q&A, that kind of thing. Is there a practical size recommendation for someone who isn't doing anything exotic, just wants a capable local model that doesn't require a supercomputer? And is there an advantage to having all this work with Claude somehow to broaden what is currently capable?

Re: Google releases Gemma 4 open models

#469

Post by lubitelpospat »

If you're using litert-lm on a Mac with Apple Silicon - DO NOT forget to use "--backend gpu"! On my M1 Pro laptop this single setting resulted in 10x prefill performance and 2x decode performance. To anyone who knows how the internals of litert-lm work - what quantization does it use? How come the model is just 3.4 GB in size?

EDIT: typo fix.

Re: Google releases Gemma 4 open models

#470

Post by tjwebbnorfolk »

canyon289 wrote: post #79
tjwebbnorfolk wrote: post #74

Earlier quoted context omitted.

Will larger-parameter versions be released?

We are always figuring out what parameter size makes sense. The decision is always a mix between how good we can make the models from a technical aspect, with how good they need to be to make all of you super excited to use them. And its a bit of a challenge what is an ever changing ecosystem. I'm personally curious is there a certain parameter size you're looking for?

All of gemma's main competitors have larger models in the 80-240b range that take advantage of larger VRAM GPUs and dual-GPU setups.

Personally I have 2x RTX 6000 PROs and right now am running the 235b-parameter Qwen model with very good results. I also occasionally use gpt-oss:120b. I would like to see a gemma model in the same range.

Also many people are running these on Mac Minis now with 128GB+ of unified RAM.

Aiming for the "runs on a single H100" tagline doesn't make a lot of sense to me, because most people do not have H100s anyway.

Post reply on HN