Second, both signal and matrix collect too much metadata. signal means you're completely screwed by their dependency on phone numbers. I expect little metadata privacy from signal because to me, it is practically the same as using my SSN or fingerprint as my user name,same for all my contacts, this key field is used by everyone and their mother to track everything we do like 1984 was target practice. For matrix, it's the defaults and how easy it is for others to fingerprint you using your specific device (equivalent of a user-agent seen by everyone iirc?) and other profile details ,but none of this is easy to correlate and answer questions like "which social network demographic micro-group does this user belong to so we can perform targeted infiltration of their device?" or "Hey, let's use this phone number to look perform the equivalent of a background check on this person who is sending us a message because we now have their phone number". Oh and the best part is, you can't just get a burner to use for registration, and to link a signal desktop,you need the mobile app. Matrix has none of these issues.
Third, consider your threat models carefully. As an individual, is it better if you have infrasructure diversity and protocol interoperability or is it better to put all your eggs in signal's basket. I never liked their use of google infra at the begining for example because I consider google more of a threat to me than most other parties. I can see the argument both ways. I personally consider the set of parties that have the most to benefit from targeting me as an individual plus those who have the most to benefit from dragnet surveillance where I reside. To me, matrix is more flexible to adopt to various threat models by for example self hosting compared to using a popular matrix server. Signal is better than the competition, if your fear is being exposed to unpatched vulnerabilities and/or if you are worried about metadata snooping (but you trust signal's infra provider, still google??) Then Signal makes more sense. For dragnet, I think matrix is better for me because implementation vulns only apply to a few users,making reliable dragnet attacks less likely. For anyone that might target me, my mobile phone is completely defenseless, so my concern is someone identifying my specific device for targeted attacks, with matrix they need to compromise the matrix server and even then they might need to do a lot more work to correlate which matrix user is me (real life "target worthy" identity). Where as with signal,they can easily micro target a group ,find out everyone's phone numbers (e.g.: hk protesters) and target their device for further exploitation via signal or any other pwnable app that is known to present on a device associated with that phone number.Practically, I am more worried about how each app fits in with everything else I do and matrix wins the security round for me.
Last but not least, I use signal for 98% of my comms because the phone number usage by Signal means I can easily connect to and invite people who don't have signal. If there is a Matrix client app that can be used as an sms client and can discover contacts' matrix account/server over sms without communicating or collecting phone number/name details of the contact, I think i might jump ship. The way I envision this to work is: the matrix client would have an invite button for non-matrix contacts and it will have an option to initiate discovery of contacts. Both options would do a challenge-response with each contact and instead of associating with a phone number they would ask to create a new martrix only contact.