Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 Technical Report [pdf]

storage.googleapis.com

Post reply on HN
Print view View original

251–260 of 260 posts

Re: Gemma 3 Technical Report [pdf]

#251

Post by setgree »

mtrovo wrote: post #239
jhayward wrote: post #225

Earlier quoted context omitted.

Two reasons it matters: 1) Discoverability 2) "System structure mirrors organization". I.E., it's an indicator of a fragmented and disorganized structure that's not likely to produce cohesive product results.

> System structure mirrors organization You listed: - one static pdf file stored on a CDN - one company blog static website - one developer documentation static website - one interactive product URL As much as I like to dunk on how messy things can be at Google I don't think this is a really good example. Apart from small startups I would be scared if you served all of them from the same base host.

The many domains is a problem because it suggests a many-teams approach to product development, and the more cooks in the kitchen, the more likely a repeat of Gemini 1’s rollout, which was a mess [0]. Basically I’m looking to see that Google cares about the meta-level user experience of finding, understanding, and using its products, and scattering key usage details around the internet is not a good sign. It suggests deeper process problems if a simple issue like this either didn’t get noticed or can’t get fixed.

[0] https://www.theguardian.com/technology/2024/mar/08/we-defini...

Re: Gemma 3 Technical Report [pdf]

#252

Post by remuskaos »

heinrichf wrote: post #111
alekandreev wrote: post #33

Greetings from the Gemma team! We just got Gemma 3 out of the oven and are super excited to show it to you! Please drop any questions here and we'll answer ASAP. (Opinions our own and not of Google DeepMind.) PS we are hiring: https://boards.greenhouse.io/deepmind/jobs/6590957

I'm comparing Gemma3 12 B ( https://ollama.com/library/gemma3 ; running fully on my 3060 12GB) and Mistral Small 3 24B ( https://ollama.com/library/mistral-small ; 10% offloaded to the CPU). - Gemma3 12B: ~100 t/s on prompt eval; 15 t/s on eval - MistralSmall3 24B: ~500 t/s on prompt eval; 10 t/s on eval Do you know what different in architecture could make the prompt eval (prefill) so much slower on the 2x smaller G…

At what context sizes? I've just run the same prompt and query on my RTX3080 with openwebui as frontend.

When I set the context size to 2048 (openwebui's default), the inference is almost twice as fast as when I set it to 4096. I can't set the conext size any higher because my GPU only has 12GB of RAM and ollama crashes for larger context sizes.

Still, I find that thoroughly odd. Using the larger conetext size (4096), the GPU usage is only 50% as seen in nvtop. I have no idea why.

Re: Gemma 3 Technical Report [pdf]

#253

Post by rodiger »

jhayward wrote: post #225
klysm wrote: post #208

Earlier quoted context omitted.

I don't see how this actually matters - who cares if it it's different top level domains?

Two reasons it matters: 1) Discoverability 2) "System structure mirrors organization". I.E., it's an indicator of a fragmented and disorganized structure that's not likely to produce cohesive product results.

> "System structure mirrors organization"

Conway's Law is the general term for this concept https://en.wikipedia.org/wiki/Conway%27s_law

Re: Gemma 3 Technical Report [pdf]

#254

Post by int_19h »

saagarjha wrote: post #180
michaelt wrote: post #91

Earlier quoted context omitted.

There are very few pro-porn voices in the corporate, tie-wearing environments that have the money to train new LLMs from scratch. Oh, there are loads of porn enjoyers working in such companies - but traditional professionalism means you leave the porn at home during the work day. It is, after all, NSFW. So at the meeting where censorship decisions were being made, even a weak argument for censoring explicit content w…

Places training LLMs don’t have many people who wear ties.

It only takes one such person in a position of power.

Re: Gemma 3 Technical Report [pdf]

#256

Post by genewitch »

genewitch wrote: post #244
Tepix wrote: post #19

Very cool to see two promising new LLMs on the same day (the other one being Reka Flash 3 21b) with open weights. Now, bring on those multimodal LLMs with voice input and output please!

OpenAI whisper and toirroise TTS. Some backends allow tool calling.

tortoise-tts*

Re: Gemma 3 Technical Report [pdf]

#257

Post by jug »

jampekka wrote: post #101

Per quick testing the 27b model seems very strong at least in natural language. It produces even good Finnish, in which smaller models tend to really struggle. Very promising. Edit: Per even quicker testing the Finnish language performance degrades rapidly with the smaller models, as is usually the case. Would be great to have language specific distillations from larger models.

I’ve hoped for this too, but as a Swede. There’s been GPT-SW3 but it was poor. We could technically have very powerful, small language specific models. I think its unfortunately just a funding and resource issue.

Re: Gemma 3 Technical Report [pdf]

#258

Post by Hiskias »

derbaum wrote: post #12
meetpateltech wrote: post #2

Gemma 3 is out! Multimodal (image + text), 128K context, supports 140+ languages, and comes in 1B, 4B, 12B, and 27B sizes with open weights & commercial use. Gemma 3 model overview: https://ai.google.dev/gemma/docs/core Huggingface collection: https://huggingface.co/collections/google/gemma-3-release-67... ollama: https://ollama.com/library/gemma3

The ollama page shows Gemma 27B beating Deepseek v3 and o3-mini on lmarena. I'm very excited to try it out.

Same!

Re: Gemma 3 Technical Report [pdf]

#259

Post by eternityforest »

canyon289 wrote: post #87
miki123211 wrote: post #73

Earlier quoted context omitted.

How good is Gemma at structured output generation, JSON schema compliance and tool use? Particularly the smaller versions, particularly in foreign languages? We will run our internal evals on it for sure, but just wanted to ask whether that's even a use case that the team considered and trained for.

Hey, I'm from the Gemma team. There's a couple of angles to your question We do care about prompted instructions, like json schema, and it is something we eval for and encourage you to try. Here's an example from Gemma2 to guide folks looking to do what it sounds like you're interested in. https://www.youtube.com/watch?v=YxhzozLH1Dk Multilinguality was a big focus in Gemma3. Give it a try And for structured output Ge…

I notice in my (brief and probably user error filled, I'm an embedded dev, not an AI expert) testing, it(and pretty much every other small model) seems to have trouble interpreting numbers expressed as words when filling out a JSON object like:

{"operator": "*", "command": "calculate", "a": 473, "b": 2848}

You might say something like five thousand fifty six, and it will fill in something like 556 or 5560.

Like as if it is just transferring digits one by one, not using the structure to know about the implicit zero.

Which is very interesting since that seems like a mistake I would make too!

It doesn't do it all the time, and I only know about the ollama quantized version, and I mostly only try the 1B models, and I've seen similar issues with other sub-2B models as well.

The other interesting thing is in a chat, almost every model I've tried seems to interpret the numbers correctly, if you say "what's ten million and fifty times eight" it will start with "10,000,050 x 8 is...".

Sometimes they get the math wrong after that, but the number interpretation is right.

I wonder if there's something special about all "intro text" in the chat mode that is actually acting like reasoning, or if the digit separators(that don't exist in JSON) help them figure out what they're doing?

I wonder if it would be better for some applications to include a line of thoughts/summary/intro in the JSON format constraint?

Other than that I've been really enjoying Gemma3!

Re: Gemma 3 Technical Report [pdf]

#260

Post by pinglin »

werediver wrote: post #240
alekandreev wrote: post #33

Greetings from the Gemma team! We just got Gemma 3 out of the oven and are super excited to show it to you! Please drop any questions here and we'll answer ASAP. (Opinions our own and not of Google DeepMind.) PS we are hiring: https://boards.greenhouse.io/deepmind/jobs/6590957

Is speculative decoding possible across 1/4/12/27 B Gemma 3 variants? LM Studio doesn't allow that (yet), but maybe the s/w requires some adjustments to support speculative decoding with Gemma 3.

It's reported working but not with LM Studio: https://www.reddit.com/r/LocalLLaMA/comments/1j9reim/comment...
Post reply on HN