Viewing profile — avisoori1x

Please note that this is borrows heavily from GIT/ Kosmos from Microsoft and LLaVA

Updated link here: https://avisoori1x.github.io/2024/04/22/seemore-_Implement_a... Hugging Face community blogs seems to have gone down.

I implemented a vision language model consisting of an image encoder, a multimodal projection module and a decoder language model in pure PyTorch. Think of this as a simplified ver…

This repo I created and the linked blog will help in understanding this: https://github.com/AviSoori1x/makeMoE

This is awesome! Thanks for sharing. I'll definitely check this out.

Quite honestly not in my experiments. I wanted to do some Bayesian hyperparameter optimization with some discretized options like noise/no-noise and n_expert/top_k but haven't been…

Oh nice. What's new here would be noisy top-k routing and expert capacity. It also seems to use the nanoGPT base from Andrej Karpathy. Mine is from January as well. Here's the orig…

This is a good point. I'm yet to try it as I've kind of let this project sit for a couple of months and only getting back to it. I went with this because it's simpler but I'm not s…

Thanks! So this is something I tried and qualitatively I didn't see a huge difference. I'd like to swap out my hand rolled modules with standard pytorch modules for self attention …

A from scratch implementation of a sparse mixture of experts language model in a single file of PyTorch. This is inspired by and largely based on Andrej Karpathy's project 'makemor…

This seems very useful. Nice work as always!