Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google releases Gemma 4 open models

deepmind.google

Post reply on HN
Print view View original

441–450 of 507 posts

Re: Google releases Gemma 4 open models

#441

Post by tr33house »

logicallee wrote: post #415
downrightmike wrote: post #287

Earlier quoted context omitted.

Did you try it?

yes, I've now I tried both the 20 GB version (gemma4:31b) which is the largest on the page[1], and the ~10 GB version (gemma4:e4b). The 20 GB version was rather slow even when fully loaded and with some RAM still left free, and the 10 GB version was speedy. I installed openclaw but couldn't get it to act as an agent the way Claude Code does. If you'd like to see a video of how both of them perform with almost nothing…

Thank you for the video. It was super helpful. the 20g version was clearly struggling but the 10g version was flying by. I think it was probably virtualized memory pages that were actually on disk causing the issue. Perhaps that and the memory compression.

Re: Google releases Gemma 4 open models

#442

Post by danielhanchen »

smallerize wrote: post #264
danielhanchen wrote: post #21

Earlier quoted context omitted.

We're still updating it haha! Sorry! It's been quite complex to support new models without breaking old ones

Speaking of which, do you think Step 3.5 Flash is going to happen or should I stop holding my breath?

Oh quants - haha I can re-investigate it - just totally forgot about them

Re: Google releases Gemma 4 open models

#443

Post by danielhanchen »

sixhobbits wrote: post #414
danielhanchen wrote: post #2

Thinking / reasoning + multimodal + tool calling. We made some quants at https://huggingface.co/collections/unsloth/gemma-4 for folks to run them - they work really well! Guide for those interested: https://unsloth.ai/docs/models/gemma-4 Also note to use temperature = 1.0, top_p = 0.95, top_k = 64 and the EOS is " ". " thought\n" is also used for the thinking trace!

Thanks for this, I gave this guide to my Claude and he oneshot the unsloth and gemma4 set up on the old macbook he runs on. It's way faster than I expected, haven't tried out local models for a few generations but will be very nice when they become useful

Thanks! Oh nice! Ye local models are advancing much faster than I expected!

Re: Google releases Gemma 4 open models

#444

Post by danielhanchen »

genpfault wrote: post #367
danielhanchen wrote: post #2

Thinking / reasoning + multimodal + tool calling. We made some quants at https://huggingface.co/collections/unsloth/gemma-4 for folks to run them - they work really well! Guide for those interested: https://unsloth.ai/docs/models/gemma-4 Also note to use temperature = 1.0, top_p = 0.95, top_k = 64 and the EOS is " ". " thought\n" is also used for the thinking trace!

llama.cpp (b8642) auto-fits ~200k context on this 24GB RX 7900 XTX & it shows a solid 100+ tok/s ("S_TG t/s") on the first 32k of it, nice! ./llama-batched-bench -hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_XL \ -npp 1000,2000,4000,8000,16000,32000,64000,96000,128000 -ntg 128 -npl 1 -c 0 | PP | TG | B | N_KV | T_PP s | S_PP t/s | T_TG s | S_TG t/s | T s | S t/s | |-------|--------|------|--------|----------|----------…

Oh nice that's pretty good!

Re: Google releases Gemma 4 open models

#445

Post by danielhanchen »

pentagrama wrote: post #385
danielhanchen wrote: post #210

Earlier quoted context omitted.

Apologies we just fixed it!! If you try again from source ie irm https://unsloth.ai/install.ps1 | iex it should work hopefully. If not - please at us on Discord and we'll help you! The Network error is a bummer - we'll check. And yes we're working on a .exe!!

It worked! https://imgur.com/a/SOfiRhv Thanks, will check it out tomorrow. Hope the unsloth-setup.exe > Windows App is coming soon! I think it will expand accessibility and user base.

Oh nice! Glad it worked! Yes!! We're working on the app!

Re: Google releases Gemma 4 open models

#449

Post by irishcoffee »

evilelectron wrote: post #57
danielhanchen wrote: post #2

Thinking / reasoning + multimodal + tool calling. We made some quants at https://huggingface.co/collections/unsloth/gemma-4 for folks to run them - they work really well! Guide for those interested: https://unsloth.ai/docs/models/gemma-4 Also note to use temperature = 1.0, top_p = 0.95, top_k = 64 and the EOS is " ". " thought\n" is also used for the thinking trace!

Daniel, your work is changing the world. More power to you. I setup a pipeline for inference with OCR, full text search, embedding and summarization of land records dating back 1800s. All powered by the GGUF's you generate and llama.cpp. People are so excited that they can now search the records in multiple languages that a 1 minute wait to process the document seems nothing. Thank you!

> your work is changing the world

I realize this may have been hyperbole, but it sure isn't changing the world.

Re: Google releases Gemma 4 open models

#450

Post by davidbjaffe »

philipkglass wrote: post #252
polishdude20 wrote: post #231

Earlier quoted context omitted.

How do you extract the content? OCR? Pdf to text then feed into qwen? I tried something similar where I needed a bunch of tables extracted from the pdf over like 40 pages. It was crazy slow on my MacBook and innacurate

If you have a basic ARM MacBook, GLM-OCR is the best single model I have found for OCR with good table extraction/formatting. It's a compact 0.9b parameter model, so it'll run on systems with only 8 GB of RAM. https://github.com/zai-org/GLM-OCR Use mlx-vlm for inference: https://github.com/zai-org/GLM-OCR/blob/main/examples/mlx-de... Then you can run a single command to process your PDF: glmocr parse example.pdf Load…

Cool! For GLM-OCR, do you use "Option 2: Self-host with vLLM / SGLang" and in that case, am I correct that there is no internet connection involved and hence connection timeouts would be avoided entirely?
Post reply on HN