Viewing profile — metalwhale

Hello everyone! I'm writing a blog about my experience on training a minimal DDPM and just want to share what I've learned so far. Feel free to read and discuss with me.

Haha I feel so dumb now. Thank you!

Thank you for sharing this! I have one question: Is there any plan to add support for local LLM / embeddings models?

I think it cannot surpass SOTA in some LM evaluation sets, but please understand that achieving better results requires a very good training dataset, which not everyone can afford.…

This is interesting. How can you retrieve the latent representation of the candidate LLMs? Some models do not have open weights (such as GPT-4), which means AFAIK it is impossible …

Thank you. Can I ask a question? Does this mean LLMs can generate text with empty context? How can LLMs choose the first token without any previous tokens? My understanding is that…

AFAIK to be called "generative", a model should have the ability to learn the joint probability over the training data. In the case of LLMs, we apply the chain rule of Bayes' formu…

Disclaimer: I'm not the author. Just want to share this awesome project.

Thank you so much for sharing this great repo! I have noticed that the source transformation notebook is not finished yet. How is it now?