Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemma 3 270M: Compact model for hyper-efficient AI

developers.googleblog.com

Post reply on HN
Print view View original

101–110 of 325 posts

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#101

Post by AJRF »

I've got a very real world use case I use DistilBERT for - learning how to label wordpress articles. It is one of those things where it's kind of valuable (tagging) but not enough to spend loads on compute for it.

The great thing is I have enough data (100k+) to fine-tune and run a meaningful classification report over. The data is very diverse, and while the labels aren't totally evenly distributed, I can deal with the imbalance with a few tricks.

Can't wait to swap it out for this and see the changes in the scores. Will report back

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#102

Post by yomismoaqui »

jefb wrote: post #64

My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…

Evaluating a 270M model on encyclopedic knowledge is like opening a heavily compressed JPG image and saying "it looks blocky"

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#103

Post by canyon289 »

gnulinux wrote: post #78
jefb wrote: post #64

My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…

Well, this is a 270M model which is like 1/3 of 1B parameters. In the grand scheme of things, it's basically a few matrix multiplications, barely anything more than that. I don't think it's meant to have a lot of knowledge, grammar, or even coherence. These input: ``` Customer Review says: ai bought your prod-duct and I wanna return becaus it no good. Prompt: Create a JSON object that extracts information about this…

Because there is a simultaneous need out of the box generalized models. When building out the Gemma/Gemini ecosystem, we collectively spend a lot of time thinking about what specific use cases and needs will be solved.

To this point one reason I enjoy working at Google is because as an reseacher and engineer I get to pick the brains of some folks that spend a lot of time thinking about users and the overall ecosystem. Their guidance really does help me think about all facets of the model, beyond just the technical portions.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#104

Post by canyon289 »

megaman821 wrote: post #96
canyon289 wrote: post #89

Earlier quoted context omitted.

We obviously need to create a pelican bicycle svg finetune ;) If you want to try this out I'd be thrilled to do it with you, I genuinely am curious how well this model can perform if specialized on that task. A couple colleagues of mine posted an example of finetuning a model to take on persona's for videogame NPCs. They have experience working with folks in the game industry and a use case like this is suitable for…

What size of tasks can this handle? Can you do a fine-tune of Mac System Settings?

32k context window so whatever fits in there. What is a finetune of mac system settings?

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#106

Post by _1 »

canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

> and with the goal you'll all finetune it for your use case.

What use-cases are a good fit for finetuning this model? More specific instruction following, knowledge from proprietary data, response tone?

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#107

Post by whymauri »

canyon289 wrote: post #95
whymauri wrote: post #67

Earlier quoted context omitted.

LLMs are really annoying to use for moderation and Trust and Safety. You either depend on super rate-limited 'no-moderation' endpoints (often running older, slower models at a higher price) or have to tune bespoke un-aligned models. For your use case, you should probably fine tune the model to reduce the rejection rate.

Speaking for me as an individual as an individual I also strive to build things that are safe AND useful. Its quite challenging to get this mix right, especially at the 270m size and with varying user need. My advice here is make the model your own. Its open weight, I encourage it to be make it useful for your use case and your users, and beneficial for society as well. We did our best to give you a great starting po…

To be fair, Trust and Safety workloads are edgecases w.r.t. the riskiness profile of the content. So in that sense, I get it.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#108

Post by littlestymaar »

yomismoaqui wrote: post #102
jefb wrote: post #64

My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…

Evaluating a 270M model on encyclopedic knowledge is like opening a heavily compressed JPG image and saying "it looks blocky"

What I read above is not an evaluation on “encyclopedic knowledge” though, it's a very basic a common sense: I wouldn't mind if the model didn't know the name of the biggest mountain on earth, but if the model cannot grasp the fact that the same mountain cannot simultaneously be #1, #2 and #3, then the model feels very dumb.

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#109

Post by canyon289 »

_1 wrote: post #106
canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

> and with the goal you'll all finetune it for your use case. What use-cases are a good fit for finetuning this model? More specific instruction following, knowledge from proprietary data, response tone?

Any text to text use case with 32k context, especially if you're starting from the PT version you can finetune it to do whatever you need

Re: Gemma 3 270M: Compact model for hyper-efficient AI

#110

Post by blitzar »

canyon289 wrote: post #2

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you. From our side we designed these models to be strong for their size out of the box, and with the goal you'll all finetune it for your use case. With the small size it'll fit on a wide range of hardware and cost…

> I built these models with a great team ... I built these models with a great team

If Gemini is going to repeat something at least its that the team is great, and not a disgrace!

Post reply on HN