I had to scroll down a shockingly long way to find this.
The point is that even if Signal permanently stored everything you ever sent them, then they wouldn't be able to read it.
- You can build the client yourself per Signal's reproducible builds, so actually, they could not ship a different app to the published source without it being immediately detectable
- You can validate the source code does not send any unencrypted data to Signal
- You can validate that your private keys used for encryption are stored locally on the device and not transmitted to Signal
Theoretically, anyone who has the corresponding private key could decrypt the message. So if your contact uses an unofficial client which does share their private key with a third party, then that third party could unencrypt that message, however by that point, the app creator has compromised the device anyway, and could do something as naive as take screenshots of all of the messages in the background after Signal has done the work of ensuring the secure transmission of the message.
Note, I haven't actually done all that, because I do trust Signal. But I could if I wanted to. And obviously, this assumes that all the cryptographic standards used in Signal are still unbroken - but if they were, then you're screwed either way.