The math behind RSA encryption

Needless to say that textbook RSA is insecure, so please, for educational purposes only ;) If you want to learn why, I can advise "Introduction to modern cryptography" by Lindell and Katz, or "Modern cryptography" by W. Mao.