Earlier quoted context omitted.
ML is not memorization. Besides, how much memory do you think this model has?
ML is a kind of memorization, though.
Anything can be a kind of something since that's subjective...
Gemma 3n preview: Mobile-first AI
41–50 of 179 posts
Re: Gemma 3n preview: Mobile-first AI
#42On Hugging face I see 4B and 2B versions now -
https://huggingface.co/collections/google/gemma-3n-preview-6...
Gemma 3n Preview
google/gemma-3n-E4B-it-litert-preview
google/gemma-3n-E2B-it-litert-preview
Interesting, hope it comes on LMStudio as MLX or GGUF. Sparse and or MoE models make a difference when running on localhost. MoE Qwen3-30B-A3B most recent game changer for me. Activating only 3b weights on the gpu cores of sparse Qwen3-30B-A3B, rather than comparable ~30b of dense models (Qwen3-32B, Gemma3-27b, GLM-{4,Z1}-32B, older QwQ-32B), is a huge speedup for me: MoE A3B achieves 20-60 tps on my oldish M2 in LMStudio, versus only 4-5 tps for the dense models.
Looking forward to trying gemma-3n. Kudos to Google for open sourcing their Gemmas. Would not have predicted that the lab with "open" in the name has yet to release even v1 (atm at 0; disregarding gpt-2), while other labs, more commercial labs, are are at versions 3, 4 etc already.
Re: Gemma 3n preview: Mobile-first AI
#43Earlier quoted context omitted.
That seems way too good to be true. What's the catch?
I used to defend LMSys/Chatbot Arena a lot but threw in the towel after events of the past three months. I can give more details if you (or anyone else!) is interested. TL;DR: it is scoring only for "How authoritative did the answer look ? How much flattering & emojis?"
Is this not what Style Control (which IIRC they're making default soon) aims to mitigate?
Re: Gemma 3n preview: Mobile-first AI
#44What is "Per Layer Embeddings"? The only hit I can find for that term is the announcement blogpost. And for that matter, what is >mix’n’match capability in Gemma 3n to dynamically create submodels It seems like mixture-of-experts taken to the extreme, where you actually create an entire submodel instead of routing per token?
You can read this for a comprehensive deep dive. https://arxiv.org/pdf/2502.01637
At a very high level, instead of having embeddings at the input layers, this method keeps the embeddings at the layer level. That is every transformer layer would have its own set of learnable embedding vectors that are used to modify the processed hidden states flowing through the network. Mostly, the embeddings are precomputed and stored separately. They are queried at inference time and has very low latency, so you can get comparable performance with half the RAM. (i am not exactly sure how 3n is doing it, but talking it in a general sense).
Re: Gemma 3n preview: Mobile-first AI
#45What is "Per Layer Embeddings"? The only hit I can find for that term is the announcement blogpost. And for that matter, what is >mix’n’match capability in Gemma 3n to dynamically create submodels It seems like mixture-of-experts taken to the extreme, where you actually create an entire submodel instead of routing per token?
I think that it's a poorly named reference to this paper [1] that they mention in the blogpost. If I had to give it another more descriptive name, I would probably name it "Per-Layer Embedding Dimensionality" [1] https://arxiv.org/pdf/2310.07707
The MatFormer is clearly called out as a different aspect of the model design.
PLE is much more likely to be a reference to the Per-Layer Embeddings paper that will be published in the future once it doesn't give away any secret sauce anymore.
Re: Gemma 3n preview: Mobile-first AI
#46Earlier quoted context omitted.
Sure, if you still think the word has meaning.
Yes, I do. Any way you slice this term, it looks close to what ML models are learning through training. I'd go as far as saying LLMs are meaning made incarnate - that huge tensor of floats represents a stupidly high-dimensional latent space, which encodes semantic similarity of every token, and combinations of tokens (up to a limit). That's as close as reifying the meaning of "meaning" itself as we ever come. (It's f…
[flagged]
Re: Gemma 3n preview: Mobile-first AI
#47Earlier quoted context omitted.
I used to defend LMSys/Chatbot Arena a lot but threw in the towel after events of the past three months. I can give more details if you (or anyone else!) is interested. TL;DR: it is scoring only for "How authoritative did the answer look ? How much flattering & emojis?"
Is this not what Style Control (which IIRC they're making default soon) aims to mitigate?
I'm not 100% sure what their rationale is for it, the launch version of style control was a statistical model that penalized a few (4?) markdown shibboleths (lists, headers, ?).
Not sure if they've shared more since.
IMVHO it won't help, at all, even if they trained a perfect model that could accurately penalize it*
The main problem is its one off responses, A/B tested. There's no way to connect it into all the stuff we're using to do work these days (i.e. tools / MCP servers), so at this point its sort of skipping the hard problems we'd want to see graded.
(this situation is a example: whats more likely, style control is a small idea for an intractable problem, or Google has now released multiple free models better than Sonnet, including the latest, only 4B params?
To my frustration, I have to go and bench these things myself because I have an AI-agnostic app I build, but I can confirm it is not the case that Gemma 3-not-n is better than Sonnet. 12B can half-consistently make file edits, which is a major step forward for local tbh)
* I'm not sure how, "correctness" is a confounding metric here: we're probably much more likely to describe a formatted answer in negative terms if the answer is incorrect.
In this case I am also setting aside how that could be done, just saying it as an illustration of no matter what, it's the wrong platform for a "how intelligent is this model?" signal, at this point, post-Eliza post-Turing, couple years out from ChatGPT 1.0
Re: Gemma 3n preview: Mobile-first AI
#48MatFormer enables pareto-optimal elasticity during inference time -- so free models between E2B and E4B as and when we need it!
Re: Gemma 3n preview: Mobile-first AI
#49What is "Per Layer Embeddings"? The only hit I can find for that term is the announcement blogpost. And for that matter, what is >mix’n’match capability in Gemma 3n to dynamically create submodels It seems like mixture-of-experts taken to the extreme, where you actually create an entire submodel instead of routing per token?
You can read this for a comprehensive deep dive. https://arxiv.org/pdf/2502.01637 At a very high level, instead of having embeddings at the input layers, this method keeps the embeddings at the layer level. That is every transformer layer would have its own set of learnable embedding vectors that are used to modify the processed hidden states flowing through the network. Mostly, the embeddings are precomputed and sto…
The paper you link to is about a different way to create embeddings at the input layer. In no way does it match your claimed description.
Re: Gemma 3n preview: Mobile-first AI
#50> Gemma 3n enables you to start building on this foundation that will come to major platforms such as Android and Chrome.
Seems like we will not be able to run this with Llama and friends.