> this model is not designed for complex conversational use cases
... but it's also the perfect choice for creative writing ...?
Isn't this a contradiction? How can a model be good at creative writing if it's no good at conversation?
Gemma 3 270M: Compact model for hyper-efficient AI
151–160 of 325 posts
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#152> this model is not designed for complex conversational use cases ... but it's also the perfect choice for creative writing ...? Isn't this a contradiction? How can a model be good at creative writing if it's no good at conversation?
Not necessarily. Where do you think the overlap is between these two tasks?
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#153Interesting that for these small models, it is optimal for the embedding parameters to be a huge fraction of the total (170e6/250e6) = 68%!
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#154Earlier quoted context omitted.
He? I know some Gemmas and it's distinctly a female name; is Gemma a boy's name where you're from?
Doesn’t the “M” in “Gemma 3 270M” Stand for “male”? Also: https://en.wikipedia.org/wiki/Gemma_Frisius
Not sure if that’s a serious question but it stands for “million”. As compared to 1B+ models, where the B stands for “billion” parameters.
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#155My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…
But who's on third?
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#156Earlier quoted context omitted.
32k context window so whatever fits in there. What is a finetune of mac system settings?
The finetune would be an LLM where you say something like "my colors on the screen look to dark" and then it points you to Displays -> Brightness. It feels like a relatively constrained problem like finding the system setting that solves your problem is a good fit for a tiny LLM.
This would be a great experiment. I'm not sure how the OS integration would work, but as a first pass you could try finetuning the model to take natural language "my colors on the screen look to dark" and then have it output "Displays -> Brightness", then expand to the various other paths you would like the model to understand
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#157Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…
Awesome! I’m curious how is the team you built these models with? Is it great?
Its hard to tell over the web whether things are sarcastic or not so excuse me if I misread the intent.
At Google I've found my colleagues to be knowledgeable, kind, and collaborative and I enjoy interacting with them. This is not just the folks I worked on this project with, but previous colleagues in other teams as well. With this particular product I've been impressed by the technical knowledge folks I worked directly with, and their contribution both improved the model's capability and my own.
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#158Earlier quoted context omitted.
I’m not sure what I’d use them for, except maybe tag generation? Encoders of this size usually outperform by a wide margin on tasks they would overlap with.
I'm making an app where literally all I want to do with an LLM is generate tags. This model has failed with flying colours, literally takes forever to parse anything and doesn't follow instructions. Edit - I should add, currently the model I'm using is Gemini Flash Lite through the Gemini API. It's a really good combo of fast, follows instructions, gives correct results for what I want and cost-effective. I still wou…
Well, Gemini Flash Lite is at least one, or likely two orders of magnitude larger than this model.
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#159This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…
It's gonna be a customer service agent for Sirius Cybernetics. Share and enjoy!
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#160Earlier quoted context omitted.
Good call, I'm trying that one just now in LM Studio (by clicking "Use this model -> LM Studio" on https://huggingface.co/unsloth/gemma-3-270m-it-GGUF and selecting the F16 one). (It did not do noticeably better at my pelican test). Actually it's worse than that, several of my attempts resulted in infinite loops spitting out the same text. Maybe that GGUF is a bit broken?
Oh :( Maybe the settings? Could you try temperature = 1.0, top_k = 64, top_p = 0.95, min_p = 0.0
Daniel, thanks for being here providing technical support as well. Cannot express enough how much we appreciate your all work and partnership.