Gemma 3 270M re-implemented in pure PyTorch for local tinkering
1–10 of 62 posts
Re: Gemma 3 270M re-implemented in pure PyTorch for local tinkering
#2[dead]
Re: Gemma 3 270M re-implemented in pure PyTorch for local tinkering
#3Hey all,
I created this model with a top notch team. I answered many questions last week when this hit the front page, and happy to answer more here as well.
https://news.ycombinator.com/item?id=44902148
Personally I'm excited that you all have access to this model now and hope you all get value out of using them.
Re: Gemma 3 270M re-implemented in pure PyTorch for local tinkering
#4Hey all, I created this model with a top notch team. I answered many questions last week when this hit the front page, and happy to answer more here as well. https://news.ycombinator.com/item?id=44902148 Personally I'm excited that you all have access to this model now and hope you all get value out of using them.
[deleted]
Re: Gemma 3 270M re-implemented in pure PyTorch for local tinkering
#5Do you think these very small models have some utility in the real world? Apart from learning and academic purposes of course.
Re: Gemma 3 270M re-implemented in pure PyTorch for local tinkering
#6Hey all, I created this model with a top notch team. I answered many questions last week when this hit the front page, and happy to answer more here as well. https://news.ycombinator.com/item?id=44902148 Personally I'm excited that you all have access to this model now and hope you all get value out of using them.
I imagine you and your team have finetuned the model on different tasks, can you share some results? (I have only seen the alien NPC finetuning)
Re: Gemma 3 270M re-implemented in pure PyTorch for local tinkering
#7Do you think these very small models have some utility in the real world? Apart from learning and academic purposes of course.
Sure, interacting with natural language without expectation that the model contains knowledge. Good for things like tool use and embeddings where the information is all retrieved.
Re: Gemma 3 270M re-implemented in pure PyTorch for local tinkering
#8Do you think these very small models have some utility in the real world? Apart from learning and academic purposes of course.
Yes! To me the primary value is not just as a teaching or toy model. I see a lot o value in repeatable tasks if we think about enterprise and a local fast developer model for individual usage.
Here's some examples that are inspired by previous roles I had outside of Google, where a business I was working in needed real time text processing.
This tutorials were made with Gemma versions from a year ago, but could now be recreated with Gemma 270m
https://developers.googleblog.com/en/gemma-for-streaming-ml-... https://www.youtube.com/watch?v=YxhzozLH1Dk
Re: Gemma 3 270M re-implemented in pure PyTorch for local tinkering
#9Hey all, I created this model with a top notch team. I answered many questions last week when this hit the front page, and happy to answer more here as well. https://news.ycombinator.com/item?id=44902148 Personally I'm excited that you all have access to this model now and hope you all get value out of using them.
I would like to know your thoughts on using 2/3 of such a small the model's size for embeddings. What would be different if you used a byte-level vocabulary and spent the parameter budget on transformer parameters instead? I think you would lose performance (tok/s) but might gain accuracy.
Re: Gemma 3 270M re-implemented in pure PyTorch for local tinkering
#10That’s wild that with a KV cache and compilation on the Mac CPU you are faster than on an A100 GPU.