Every peer is identified by a curve25519 public key. To add someone as a friend, you add that public key.
Connections between friends are encrypted.
http://nacl.cr.yp.to/box.html
is the crypto used.
https://github.com/irungentoo/toxcore/blob/master/docs/Tox_m...
describes the protocol used to connect securely to friends after they find themselves.
This protocol has PFS, message padding to prevent length based leaking and should be immune against replay attacks.
What makes it secure is that you know the long term public key of your friends making it really easy for the software to establish secure connections to them.