[flagged]
Google releases Gemma 4 open models
401–410 of 507 posts
Re: Google releases Gemma 4 open models
#402[flagged]
Re: Google releases Gemma 4 open models
#403[flagged]
Re: Google releases Gemma 4 open models
#404I'm curious about the multimodal capabilities on the E2B and E4B and how fast is it. In ChatGPT right now, you can have a audio and video feed for the AI, and then the AI can respond in real-time. Now I wonder if the E2B or the E4B is capable enough for this and fast enough to be run on an iPhone. Basically replicating that experience, but all the computations (STT, LLM, and TTS) are done locally on the phone. I just…
I just want to say thanks. Finding out about these kind of projects that people are working on is what I come to HN for, and what excites me about software engineering!
Thank you for the kind words!
Re: Google releases Gemma 4 open models
#405Earlier quoted context omitted.
Hey in really interested in your pipeline techniques. I've got some pdfs I need to get processed but processing them in the cloud with big providers requires redaction. Wondering if a local model or a self hosted one would work just as well.
Disclaimer: I'm an AI novice relative to many here. FWIW last wknd I spent a couple hours setting up self-hosted n8n with ollama and gemma3:4b [EDIT: not Qwen-3.5], using PDF content extraction for my PoC. 100% local workflow, no runtime dependency on cloud providers. I doubt it'd scale very well (macbook air m4, measly 16GB RAM), but it works as intended.
For those who wish to do OCR on photos, like receipts, or PDFs or anything really, Paperless-NGX works amazingly well and runs on a potato.
Re: Google releases Gemma 4 open models
#406[dead]
Re: Google releases Gemma 4 open models
#407Earlier quoted context omitted.
Hey in really interested in your pipeline techniques. I've got some pdfs I need to get processed but processing them in the cloud with big providers requires redaction. Wondering if a local model or a self hosted one would work just as well.
I run llama.cpp with Qwen3-VL-8B-Instruct-Q4_K_S.gguf with mmproj-F16.gguf for OCR and translation. I also run llama.cpp with Qwen3-Embedding-0.6B-GGUF for embeddings. Drupal 11 with ai_provider_ollama and custom provider ai_provider_llama (heavily derived from ai_provider_ollama) with PostreSQL and pgvector. People on site scan the documents and upload them for archival. The directory monitor looks for new files in…
Are you linking any of the processes using the Drupal AI module suite?
Re: Google releases Gemma 4 open models
#408Thinking / reasoning + multimodal + tool calling. We made some quants at https://huggingface.co/collections/unsloth/gemma-4 for folks to run them - they work really well! Guide for those interested: https://unsloth.ai/docs/models/gemma-4 Also note to use temperature = 1.0, top_p = 0.95, top_k = 64 and the EOS is " ". " thought\n" is also used for the thinking trace!
and the EOS is " ". " thought\n" is also used for the thinking trace! Can someone explain this to me? Why is this faux-XML important here?
These are likely individual tokens. They are super common.
Re: Google releases Gemma 4 open models
#409Earlier quoted context omitted.
We (LM Studio) found the bug with the 31B model and a fix will be going out hopefully tonight
I am not deep in this world. What does it mean when you (LM Studio) fixed a bug in a model Google released?
There is a surprising amount of code needed in each of the inference frameworks (LM Studio, llama.cpp, etc) to support each new model release. For example to format the input in the right way using a chat template, to parse the output properly with the model-specific tokens the model provider decided to standardize on for their model, and more.
This particular instance was a fix to the output parsing [1] in LM Studio, described like this:
"Adds value type parsers that use as string delimiters instead of JSON's double quotes, and disables json-to-schema conversion for these types."
[1]: https://github.com/ggml-org/llama.cpp/pull/21326/commits/a50...
edit: formatting