Viewing profile — jaymody

I wrote an article that attempts to explain attention in a more intuitive manner, without any PyTorch or other deep learning specific context: https://jaykmody.com/blog/attention-i…

I wrote a minimal implementation in NumPy here (the forward pass code is only 40 lines): https://github.com/jaymody/picoGPT And also a related blog post: https://news.ycombinator.c…