Viewing profile — nathan-barry

LLMs are very good at lossless compression via arithmetic coding. But I didn't know that it was possible to go the reverse direction (do language modeling via a compressor). It's n…

I previously posted this a month and a half ago: https://news.ycombinator.com/item?id=45876742 I have rewritten it to be much more simple, from 951 lines of code to just 364, with …

Actually NVIDIA made one earlier this year, check out their Fast-dLLM paper

Hey, I’ll add the MIT licenses later today!

Yes, this is the case. During training, the model will get a sequence of text (ex, 512 tokens long) with a percentage of them masked out (with a special token). It learns how to un…

This is a character-level language diffusion model for text generation. The model is a modified version of Nanochat's GPT implementation and is trained on Tiny Shakespeare! It is o…