Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 4 12B: A unified, encoder-free multimodal model

blog.google

Post reply on HN
Print view View original

411–420 of 421 posts

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#411

Post by mdp2021 »

bitexploder wrote: post #262
mdp2021 wrote: post #208

Earlier quoted context omitted.

> I suspect ... still wins in general world knowledge due to bigger size Encyclopedic knowledge matters relatively little in perspective, given the expectable future developments: even the more knowledgeable of us will use that knowledge for reasoning and intuition (and we will have absorbed the intellectual keys during our training), but under our professional hat we should in theory be ready to go "I stand correcte…

Don't LLMs work on attention though? The closer in their hyperdimensional space you can land your problem to their inherent understand the better they are at understanding your problem domain. RAG loops can be very slow and agents may simply lack the knowledge to use them correctly.

But, in short, the ability to manage information, to process it properly, is more important in this regard than just having the information. "Having" more knowledge is not a guarantee to "using" it better.

And to improve reliability, if the machine can check, it will have to check. "Costly" cannot be an excuse.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#412

Post by szmarczak »

powera wrote: post #147

I'm seeing very low quality results on LMStudio with this model. Worse than Gemma 3 12B. It is getting questions like "David has 18 apples and Ivan has 7 apples. How many apples do they have together?" wrong half the time, while Gemma3 12B could very consistently answer that. Other smoke tests (like Chinese translation, and the infamous "Rs in Strawberry" test) also show poor results. I don't know if it is a quantiza…

I just ran this 25 times and every time I got 25. Ran this via ollama.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#414

Post by dofm »

minimaxir wrote: post #409
dofm wrote: post #406

Earlier quoted context omitted.

I wonder if they were just slightly ahead of this announcement? https://blog.google/innovation-and-ai/technology/developers-... Looks like the 12B model should fit now?

It definitely works in LM Studio, not Edge Gallery yet.

Following up again, in case you see this. I was just in oMLX trying to set up the new 26B QAT models with MTP, and I noticed this message:

  Kernel iogpu.wired_limit_mb is only 48.0 GB; oMLX can only allocate up to 48.0 GB. Raise it in Terminal:
  sudo sysctl iogpu.wired_limit_mb=59392
Perhaps if you can increase the wired limit it will fit?

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#415

Post by dofm »

thot_experiment wrote: post #232
dofm wrote: post #220

Earlier quoted context omitted.

I don't have the energy to build stuff all the time, that's a rabbit-hole side tunnel I don't really want to get into. I have larger concerns in my life that are more urgent than developing that side of things. But I've moved on from Ollama for the time being, though I am mainly interested to see what the Gemma 4 MTP speeds are like on my M1 Max, so I may test it. I am quite impressed with the tools in LM Studio, whi…

Totally understandable. YMMV but I found the llama.cpp build process to work on the first try on my machine, and it only takes a couple minutes, which definitely isn't my usual expectation or experience. I was very pleasantly surprised. Their web-ui is also getting very polished while still doing a great job of letting you tweak all the weird settings.

I never did quite get round to a llama.cpp build but support is there now anyway and I have the MTP drafter working with the QAT 26B build:

https://news.ycombinator.com/item?id=48441450

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#416

Post by cwmoore »

altruios wrote: post #104
alberto467 wrote: post #49

Earlier quoted context omitted.

Not at all. Getting really pedantic, tokenization is also a form of encoding, so it doesn't matter the modality you're using, you'll end up doing some type of encoding in some way.

Tokens are such a strange base unit. Couldn't we do something that naturally conforms better to reality than such choppy units that cause all sorts of artifacts? making everything 'language based' prevents true multi-modality. Thinking isn't done in language. Thinking outputs language, but its far more like multiple waves of data coalescing into an 'idea', internal... subjectively (n=1) at least. I think wave/signal…

Symbolic information is tokenizable as bits is the reason.

I am intrigued by wave properties but also semi-familiar with Claude Shannon and waves can be modeled in many cases, if not all.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#417

Post by MoonWalk »

embedding-shape wrote: post #347
MoonWalk wrote: post #277

Earlier quoted context omitted.

Why the obfuscation in the first place?

Just a bit of flair. Also, bunch of people have "keyword watchers" setup for various terms, so when you mention certain things on HN, reddit and elsewhere, you get commentators who enter the conversation not because the context or larger conversation, but because the single term/thing they care deeply about was mentioned, and it just gets very boring to read the whole attackers/defenders comments over and over again.…

But it renders the comment baffling to those who have never heard of that forum. I'm on here and Reddit quite a bit, and never heard of it.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#418

Post by zigzag312 »

mdp2021 wrote: post #411
bitexploder wrote: post #262

Earlier quoted context omitted.

Don't LLMs work on attention though? The closer in their hyperdimensional space you can land your problem to their inherent understand the better they are at understanding your problem domain. RAG loops can be very slow and agents may simply lack the knowledge to use them correctly.

But, in short, the ability to manage information, to process it properly, is more important in this regard than just having the information. "Having" more knowledge is not a guarantee to "using" it better. And to improve reliability, if the machine can check, it will have to check. "Costly" cannot be an excuse.

Understanding of a specific problem space can be a prerequisite to be able to form a proper query (i.e. to ask the correct question).

Model doesn't know what it doesn't know.

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#419

Post by kristjansson »

kristjansson wrote: post #164
thomasjb wrote: post #155

Unfortunately there's no gguf quants of the assistant model yet: https://huggingface.co/models?other=base_model:quantized:goo...

I think MTP Gemma4 support is still WIP https://github.com/ggml-org/llama.cpp/pull/23398 ?

Update: not any more!

Re: Gemma 4 12B: A unified, encoder-free multimodal model

#420

Post by mdp2021 »

zigzag312 wrote: post #418
mdp2021 wrote: post #411

Earlier quoted context omitted.

But, in short, the ability to manage information, to process it properly, is more important in this regard than just having the information. "Having" more knowledge is not a guarantee to "using" it better. And to improve reliability, if the machine can check, it will have to check. "Costly" cannot be an excuse.

Understanding of a specific problem space can be a prerequisite to be able to form a proper query (i.e. to ask the correct question). Model doesn't know what it doesn't know.

Your suggestion is not clear: yes we reason and define relevant details (maybe through further information retrieval) to better construct queries - that is what Analytical school of thought taught and insisted on -, and even more crucial is that the subsequent delegated steps, of constructing replies, imply reasoning and information retrieval.

Said abilities - intellectual strength - are immensely more important than notions. The relation between network size and intellectual strength, vs network size and notions (original topic in this branch), is presumably not yet that clear. Intelligent models may not necessarily be embedded with explicit information of everything, though they will have to have ways to reach that upon contingent necessity (to solve specific problems). Like us.

Post reply on HN