As mentioned in other comments, you have to trust Apple to hand you the correct public keys. They could easily MITM you and decrypt the messages on the server if they misrepresent the other party's public key. Additionally, the iMessages you send are signed by your private key, which is probably not something you want.
Why not?