Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 Technical Report [pdf]

storage.googleapis.com

Post reply on HN
Print view View original

221–230 of 260 posts

Re: Gemma 3 Technical Report [pdf]

#221

Post by simne »

lastLinkedList wrote: post #218
npodbielski wrote: post #59

Earlier quoted context omitted.

I believe (and some other people on the internet having more knowledge in LLM believe too) that open source local models are the future. Probably big models with API and chat like OpenAI is doing will have its niche toot but it is very costly and it is not AGI and it will not be in the near future. On the other hand with rise of NPU chips and small models you can have your own assistant on your phone using your own d…

I’m curious to hear more about phone-local assistants. I rather assumed only the latest hardware ( iPhone 15+, not sure on Android side) could do local inference. Is there a way to get something going on hardware a couple years old?

> Is there a way to get something going on hardware a couple years old?

Tensor accelerators are very recent thing, and GPU/WebGPU also recent. RAM was also limited, 4Gb was long time barrier.

So, model should run on CPU and within 4Gb or even 2Gb.

Oh, I forget one important thing - couple years old mobile CPUs was also weak (and btw exception was iphone/ipad).

But, if you have gaming mobile (or iphone), which at that time was comparable to Notebooks, may run something like Llama-2 quantized to 1.8Gb at about 2 tokens per second, not very impressive, but could work.

Re: Gemma 3 Technical Report [pdf]

#223

Post by BoredPositron »

philipkglass wrote: post #206
BoredPositron wrote: post #205

Earlier quoted context omitted.

There are models especially tuned for it even open weight ones. llms even multimodal ones are not up to the task. You know what doesn't help the discussion at all? That everyone's response is as usual just about titties.

4 months ago I tried every dedicated NSFW-image-classifier model I could find on HuggingFace or GitHub. They have a high false positive rate on certain kinds of benign content, like close up photographs of hands with painted fingernails, and a high false negative rate on artistic nude photographs. I even tried combining multiple models with gradient boosting but the accuracy barely improved; maybe everyone is trainin…

You can just finetune an open model instead of starting from scratch... that's the point of them.

Re: Gemma 3 Technical Report [pdf]

#224

Post by simne »

gundmc wrote: post #37

What do companies like Meta and Google gain from releasing open models? Is it just reputational? Attractive to top AI talent?

Unfortunately, this is known business model, most known example was Eclipse IDE, which killed all small IDE businesses. Other example, MySQL from Oracle.

Yes, idea, to make basically free something, on which small-medium businesses could survive and grow to something big, so making big death valley between small and big businesses.

Only exception are tiny businesses, living in tiny niches, but for them nearly impossible to overcome gap from tiny to big.

And you should understand, "open models" are in reality open-weight models, as they not disclose sources from which trained, so community cannot remake model from scratch.

Headhunting is sure important, but big business typically are so much finance powerful, so they could just buy talents.

- Headhunting with reputation is really important for small businesses, because they typically very limited in finances.

Medium business typically between small and big, but as I said at beginning, making some strategic things free, create death valley, so it become very hard to be medium.

Reputation is good thing for all, but again, top corporations are powerful non-proportional to size, so in many cases for them is relatively cheap to just maintain neutral reputation, they don't need to spend much to whitening.

Re: Gemma 3 Technical Report [pdf]

#225

Post by jhayward »

klysm wrote: post #208
setgree wrote: post #203

Earlier quoted context omitted.

A kind of ancillary note, but it's amazing to me how fragmented this presentation and documentation is: * the parent link is to storage.googleapis.com * There's documentation on ai.google.dev * The announcement blogpost is https://blog.google/technology/developers/gemma-3/ * you try it on https://aistudio.google.com/ It's helpful to have a top-level post like this, but can some PM please consolidate this into, IDK, a…

I don't see how this actually matters - who cares if it it's different top level domains?

Two reasons it matters:

1) Discoverability

2) "System structure mirrors organization". I.E., it's an indicator of a fragmented and disorganized structure that's not likely to produce cohesive product results.

Re: Gemma 3 Technical Report [pdf]

#226

Post by simne »

simne wrote: post #221
lastLinkedList wrote: post #218

Earlier quoted context omitted.

I’m curious to hear more about phone-local assistants. I rather assumed only the latest hardware ( iPhone 15+, not sure on Android side) could do local inference. Is there a way to get something going on hardware a couple years old?

> Is there a way to get something going on hardware a couple years old? Tensor accelerators are very recent thing, and GPU/WebGPU also recent. RAM was also limited, 4Gb was long time barrier. So, model should run on CPU and within 4Gb or even 2Gb. Oh, I forget one important thing - couple years old mobile CPUs was also weak (and btw exception was iphone/ipad). But, if you have gaming mobile (or iphone), which at that…

Unfortunately, I could not remember, when median performance of mobile CPU become comparable to business Notebooks.

I think, Apple entered race for speed with iPhone X and iPad 3. For Androids things even worse, looks like median achieved Notebooks speed at about Qualcomm snapdragon 6xx.

Re: Gemma 3 Technical Report [pdf]

#227

Post by tadamcz »

ZeroCool2u wrote: post #209
tadamcz wrote: post #191

The launch post for Gemma 3 says: > use Gemma 3 with the Google GenAI SDK https://blog.google/technology/developers/gemma-3/ Does this mean (serverless) API access? I haven't been able to do so or find docs that explain how to.

You can just go here: https://aistudio.google.com/prompts/new_chat Select Gemma 3 from the drop down on the right side.

This doesn't explain how to get API access

Re: Gemma 3 Technical Report [pdf]

#228

Post by svachalek »

diggan wrote: post #151
meetpateltech wrote: post #2

Gemma 3 is out! Multimodal (image + text), 128K context, supports 140+ languages, and comes in 1B, 4B, 12B, and 27B sizes with open weights & commercial use. Gemma 3 model overview: https://ai.google.dev/gemma/docs/core Huggingface collection: https://huggingface.co/collections/google/gemma-3-release-67... ollama: https://ollama.com/library/gemma3

> open weights What exactly is this supposed to mean? That I can grab the weights by just downloading them, or something like that? Because when I open up the HuggingFace repository, it asks me to "accept the conditions" (Google’s usage license). How is this different from any other proprietary binaries people distribute on the internet but let you run locally? Are other software (like 1Password for example) also "op…

"Open weights" refers to a license that allows you to freely (or mostly freely) copy the model file (i.e. weights). An "open source" model would be possible to build from training data, but those hardly exist.

Re: Gemma 3 Technical Report [pdf]

#229

Post by refulgentis »

canyon289 wrote: post #87
miki123211 wrote: post #73

Earlier quoted context omitted.

How good is Gemma at structured output generation, JSON schema compliance and tool use? Particularly the smaller versions, particularly in foreign languages? We will run our internal evals on it for sure, but just wanted to ask whether that's even a use case that the team considered and trained for.

Hey, I'm from the Gemma team. There's a couple of angles to your question We do care about prompted instructions, like json schema, and it is something we eval for and encourage you to try. Here's an example from Gemma2 to guide folks looking to do what it sounds like you're interested in. https://www.youtube.com/watch?v=YxhzozLH1Dk Multilinguality was a big focus in Gemma3. Give it a try And for structured output Ge…

The Ollama stuff is the old llama.cpp stuff that constrains output tokens.

It's great, I've used it to get outputs from as small a model as 1B.

But it's a stark difference in quality from, say, Phi-4's native tool-calling.

If Gemma 3 is natively trained on tool-calling, i.e. y'all are benching on say, Berekley Function Calling leaderboard, that'd be great to know out here.

Tangentially, github.com/ochafik is a Googler who landed an excellent overhaul of llama.cpp's tool-calling, might be worth reaching out to (if you're not working with him already!)

Re: Gemma 3 Technical Report [pdf]

#230

Post by gslaller »

A noob speaking here. Why aren't there efforts to have a memory bank like structure where you attend to a sub set of codes depending on the key(at the attention level)? is this already done with the global attention mechanism (what is it even)?
Post reply on HN