Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google releases Gemma 4 open models

deepmind.google

Post reply on HN
Print view View original

451–460 of 507 posts

Re: Google releases Gemma 4 open models

#451

Post by evilelectron »

lwhi wrote: post #407
evilelectron wrote: post #214

Earlier quoted context omitted.

I run llama.cpp with Qwen3-VL-8B-Instruct-Q4_K_S.gguf with mmproj-F16.gguf for OCR and translation. I also run llama.cpp with Qwen3-Embedding-0.6B-GGUF for embeddings. Drupal 11 with ai_provider_ollama and custom provider ai_provider_llama (heavily derived from ai_provider_ollama) with PostreSQL and pgvector. People on site scan the documents and upload them for archival. The directory monitor looks for new files in…

Are you linking any of the processes using the Drupal AI module suite?

Yes, they are all linked using Drupal's AI modules. I have an OpenCV application that removes the old paper look, enhances the contrast and fixes the orientation of the images before they hit llama.cpp for OCR and translation.

Re: Google releases Gemma 4 open models

#452

Post by 5555watch »

XCSme wrote: post #300
canyon289 wrote: post #51

Hi all! I work on the Gemma team, one of many as this one was a bigger effort given it was a mainline release. Happy to answer whatever questions I can

Good work, it's quite close to Gemini 3 Pro in my tests, but 10x cheaper: https://aibenchy.com/compare/google-gemma-4-31b-it-medium/go...

Why no (high) variants in the comparison models?

Re: Google releases Gemma 4 open models

#454

Post by dave7 »

kristopolous wrote: post #428
zozbot234 wrote: post #259

Earlier quoted context omitted.

These are typically coding oriented as opposed to general chat, so their system prompts may be needlessly heavy for that use case. I think the closest thing to a general solution is the emerging "claw" ecosystem, as silly as that sounds. Some of the newer "claws" do provide proper sandboxing.

This is actually a perfect use case of my llcat ... Using Unix philosophy for the llm era https://github.com/day50-dev/llcat I've got an interesting hack brewing for extremely hassle free tool orchestration - basically think along the lines of .bash_profile level simplicity... Maybe I'll get that out tomorrow

llcat looks like it could be very handy, thank you!

Re: Google releases Gemma 4 open models

#455

Post by Breza »

evilelectron wrote: post #57
danielhanchen wrote: post #2

Thinking / reasoning + multimodal + tool calling. We made some quants at https://huggingface.co/collections/unsloth/gemma-4 for folks to run them - they work really well! Guide for those interested: https://unsloth.ai/docs/models/gemma-4 Also note to use temperature = 1.0, top_p = 0.95, top_k = 64 and the EOS is " ". " thought\n" is also used for the thinking trace!

Daniel, your work is changing the world. More power to you. I setup a pipeline for inference with OCR, full text search, embedding and summarization of land records dating back 1800s. All powered by the GGUF's you generate and llama.cpp. People are so excited that they can now search the records in multiple languages that a 1 minute wait to process the document seems nothing. Thank you!

I'm very active in family history and this kind of project is massively helpful, thank you

Re: Google releases Gemma 4 open models

#456

Post by notnullorvoid »

stavros wrote: post #301
neonstatic wrote: post #228

Prompt: > what is the Unix timestamp for this: 2026-04-01T16:00:00Z Qwen 3.5-27b-dwq > Thought for 8 minutes 34 seconds. 7074 tokens. > The Unix timestamp for 2026-04-01T16:00:00Z is: > 1775059200 (my comment: Wednesday, 1 April 2026 at 16:00:00) Gemma-4-26b-a4b > Thought for 33.81 seconds. 694 tokens. > The Unix timestamp for 2026-04-01T16:00:00Z is: > 1775060800 (my comment: Wednesday, 1 April 2026 at 16:26:40) Gem…

To clarify, the parent here didn't actually give the model a way to run the commands. The model just wrote the script/command and then, being unable to run anything, just mentally calculated what the result would probably be (and got it wrong). Yes the answer was wrong, but so was the setup (the model should have had access to a command runner tool).

Regardless of setup the LLM shouldn't hallucinate tool use.

Re: Google releases Gemma 4 open models

#457

Post by philipkglass »

davidbjaffe wrote: post #450
philipkglass wrote: post #252

Earlier quoted context omitted.

If you have a basic ARM MacBook, GLM-OCR is the best single model I have found for OCR with good table extraction/formatting. It's a compact 0.9b parameter model, so it'll run on systems with only 8 GB of RAM. https://github.com/zai-org/GLM-OCR Use mlx-vlm for inference: https://github.com/zai-org/GLM-OCR/blob/main/examples/mlx-de... Then you can run a single command to process your PDF: glmocr parse example.pdf Load…

Cool! For GLM-OCR, do you use "Option 2: Self-host with vLLM / SGLang" and in that case, am I correct that there is no internet connection involved and hence connection timeouts would be avoided entirely?

When you self-host, there's still a client/server relationship between your self-hosted inference server and the client that manages the processing of individual pages. You can get timeouts depending on the configured timeouts, the speed of your inference server, and the complexity of the pages you're processing. But you can let the client retry and/or raise the initial timeout limit if you keep running into timeouts.

That said, this is already a small and fast model when hosted via MLX on macOS. If you run the inference server with a recent NVidia GPU and vLLM on Linux it should be significantly faster. The big advantage with vLLM for OCR models is its continuous batching capability. Using other OCR models that I couldn't self-host on macOS, like DeepSeek 2 OCR or Chandra 2, vLLM gave dramatic throughput improvements on big documents via continuous batching if I process 8-10 pages at a time. This is with a single 4090 GPU.

Re: Google releases Gemma 4 open models

#458

Post by nickthegreek »

stavros wrote: post #344
g947o wrote: post #335

Earlier quoted context omitted.

*8 months That doesn't make parent's claim true or even relevant. And OpenAI could release an open model tomorrow. Nobody knows.

Anybody could release an open model tomorrow. Google is the only US based lab releasing open weights models. OpenAI released one once, which might or might not count as "releasing", depending on your definition

> OpenAI released one once

gpt-2, CLIP, Whisper, Point-E, got-oss-120b, gpt-oss-20b.

Re: Google releases Gemma 4 open models

#459

Post by rizzo94 »

danielhanchen wrote: post #2

Thinking / reasoning + multimodal + tool calling. We made some quants at https://huggingface.co/collections/unsloth/gemma-4 for folks to run them - they work really well! Guide for those interested: https://unsloth.ai/docs/models/gemma-4 Also note to use temperature = 1.0, top_p = 0.95, top_k = 64 and the EOS is " ". " thought\n" is also used for the thinking trace!

Huge fan of the Unsloth quants! Having reasoning and tool calling this accessible locally is a massive leap forward.

The main hurdle I've found with local tool calling is managing the execution boundaries safely. I’ve started plugging these local models into PAIO to handle that. Since it acts as a hardened execution layer with strict BYOK sovereignty, it lets you actually utilize Gemma-4's tool calling capabilities without the low-level anxiety of a hallucination accidentally wiping your drive. It’s the perfect secure gateway for these advanced local models.

Post reply on HN