Now if they would drop the ridiculous requirement of having a phone number and go with usernames and not require access to my contact list like most other services, you could actually be safer and not rely on _their word_ alone.
With the approach that Signal has taken, it doesn't require you to register a username. The app just registers you with the server as a Signal user using your already unique ID - your phone number. While not the most secure and privacy-protecting method, it allows anyone to just install it and start using it as a secure alternative to plaintext SMS.
Regarding the simplicity of using a phone number and how it would be very easy for all users - Wickr provides basically the same service, is more popular, but uses usernames instead of phone numbers.
The fact that something is open source isn't always a clear win. OpenSSL, for example, is open source and yet they seem to have many bugs.
For the record, I use Signal almost exclusively for communication. I just wish it was more protective of users' privacy without having to put in a ton of effort.