Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3n preview: Mobile-first AI

developers.googleblog.com

Post reply on HN
Print view View original

101–110 of 179 posts

Re: Gemma 3n preview: Mobile-first AI

#101

Post by int_19h »

hmapple wrote: post #60
rhdjsjebshjffn wrote: post #32

Earlier quoted context omitted.

I can't speak for anyone else, but these models only seem about as smart as google search, with enormous variability. I can't say I've ever had an interaction with a chatbot that's anything redolent of interaction with intelligence. Now would I take AI as a trivia partner? Absolutely. But that's not really the same as what I look for in "smart" humans.

Have you tried any SOTA models like o3? If not, I strongly encourage you to discuss your area of expertise with it and rate based on that It is incredibly competent

SOTA models can be pretty smart, but this particular model is a very far cry from anything SOTA.

Re: Gemma 3n preview: Mobile-first AI

#103

Post by int_19h »

lxgr wrote: post #15
onlyrealcuzzo wrote: post #14

Earlier quoted context omitted.

That sounds like a problem iOS will eventually deal with, as many apps are going to want this technology, and since Apple distributes apps - they aren't interested in the average app being 10x larger when they could solve the problem easily. Though, I won't be surprised if they try to force devs to use their models for "privacy" (and not monopolistic reasons, of course).

Given Apple's track record in dealing with the problem of ballooning app sizes, I'm not holding my breath. The incentives are just not aligned – Apple earns $$$ on each GB of extra storage users have to buy.

https://www.bloomberg.com/news/articles/2025-05-20/apple-to-...

Re: Gemma 3n preview: Mobile-first AI

#105

Post by osanseviero »

jdiff wrote: post #68
zamadatix wrote: post #37

Earlier quoted context omitted.

The link says E2B and E4B have 4B and 8B raw parameters, where do you see 7B?

There's a 7B mentioned in the chat arena ELO graph, I don't see any other references to it though.

Hi! The model is 8B if you also load the vision and audio components. We just used the text model in LMArena.

Re: Gemma 3n preview: Mobile-first AI

#106

Post by vineyardmike »

mltsd wrote: post #75

I wonder how powerful the models our phones can run will be when (if?) they figure out how to make them 'specialized', i.e. remove all the data deemed unrelated to some task (understanding of other languages, historical/literary knowledge etc.), even if hardware doesn't improve much it seems there's still a lot to optimize

I generally assume this is how existing model developers have been improving. Gemini especially is crazy fast, and thanks to google search integration, Gemini-the-model doesn’t need to “know” any trivia.

Re: Gemma 3n preview: Mobile-first AI

#107

Post by yorwba »

ankit219 wrote: post #55
yorwba wrote: post #49

Earlier quoted context omitted.

The paper you link to is about a different way to create embeddings at the input layer. In no way does it match your claimed description.

I simplified what i wrote. There is an off accelerator memory where the embeddings are stored and queried at inference time, i did not want to get into details. That is how you reduce the in memory RAM. There are definitely more things going on in the paper as it builds upon the concept I described. The central idea remains the same: you have input embedding layers which map text to continuous vectors. Instead of loa…

The name "per-layer embeddings" is all we have to go on, and there are currently no published papers (that I'm aware of) using any similar mechanism, so, yes, it's a huge leap from a paper that doesn't mention per-layer anything.

It's fine to speculate based on the name, but don't pretend that it's a known technique when it clearly isn't.

Re: Gemma 3n preview: Mobile-first AI

#108

Post by theyinwhy »

lxgr wrote: post #86
elpakal wrote: post #64

Earlier quoted context omitted.

I don't know how true your comment is about them earning money on each GB, but if you're interested in app size analysis on iOS I made this for that reason https://dotipa.app . I occasionally post decompositions of public .ipa's on the App Store, and I'm looking forward to seeing how these change over the next year.

It seems straightforward to me: Apps take up storage, and the only way to get more of that is to pay Apple's markups, as iOS devices don't support upgradable storage. On top of the already hefty markup, they don't even take storage capacity into consideration for trade-ins.

I am not aware of any phone allowing storage upgrades.

Re: Gemma 3n preview: Mobile-first AI

#109

Post by [deleted] »

nolist_policy wrote: post #35

You can try it on Android right now: Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0 Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6... Import the .task file in Edge Gallery with the + bottom right. You can take pictures right from the app. The model is indeed pretty fast.

[deleted]

Re: Gemma 3n preview: Mobile-first AI

#110

Post by krackers »

kcorbitt wrote: post #51
krackers wrote: post #8

Earlier quoted context omitted.

Thank you, that helped a bit, although it's still not clear what exactly those parameters _are_. "Per-Layer Embedding (PLE) parameters that are used during model execution to create data that enhances the performance of each model layer." is too vague, and I can't find any other reference to "per-layer embedding parameters" in literature.

I wonder if they've trained the model to operate with a shallower stack; eg. the full model may be composed of 24 transformer blocks, but they've also trained it to accept embeddings at layer 8, so it can be operated with just 16 transformer blocks on lower-resourced devices. Experimenters in the open source tinkering community have done the opposite (copy/pasting layers in existing models to make them deeper) and it…

Someone extracted out the dims in https://twitter.com/cccntu/status/1925043973170856393#m

It seems to be embedding from 262k possible vocab tokens down to 256 dims. 262144 matches the same vocab size used for the existing Gemma model, so it really does seem to be an embedding of the input token directly, fed into each layer.

I guess intuitively it might help the model somewhat for later layers to have direct access to the input query without needing to encode it in the residual stream, and it can use those parameters for something else. I'm kind of surprised no one tried this before, if the idea is that simple? Reminds me of resnet where you have the "skip" layers so future layers can access the input directly.

Edit: As for what exactly the embedding is used for, it could be that the embedding is still used for something more clever than induction head-type stuff. Responses in [1] suggest it might be some low-rank data/token dependent signal that can be "factored out"/precomputed. Another clever suggestion was that it's a per-layer input-token-derived control/steering vector.

[1] https://twitter.com/CFGeek/status/1925022266360005096

Post reply on HN