Signal is built around metadata minimisation. Messages contain the absolute bare minimum information to get delivered. Because there is only one server everyone is connected to, there is almost no routing metadata attached to the encrypted binary blob. You get a key and ciphertext and that's about it. Not even Signal knows the sender metadata that's part of the message, by encrypting it. Basically all Signal knows about you is your phone number, IP address, and the last time you checked the server. It also doesn't store messages for longer than it absolutely needs to (which leads to the desktop client needing to connect to your phone or vice versa for multi device chats).
XMPP needs to have a username and server name at the very least. This is because it needs to work in a federated context, and it doesn't use things like DHTs to decentralise messages in a way that allows hiding the routing data. The message body is encrypted, of course, and headers can be minimised, but there will always be unencrypted metadata. To quote the spec:
> The OMEMO protocol does not protect against attackers who rely on metadata and traffic analysis.
As for Matrix: the message body is usually mostly encrypted, but it's leaking a lot of metadata. Message IDs sometimes fine themselves outside of the encrypted envelope as well as timestamps and other information I don't think should need to be outside the encrypted envelope.
Neither XMPP nor Matrix were designed with encryption as a first priority and that led to protocol design choices regarding metadata that cannot be altered without breaking most clients. They also tend to store messages grouped by chat group/conversation, though multi device support is technically optional for XMPP. From a server dump of either XMPP or Matrix, someone can deduce what users are chatting to what users when. In Matrix, you could deduce what messages are responses, updates, or deletions of what other messages, as well as reactions. For Signal, you'd need wiretaps on both sides to deduce that level of information.
A protocol like Signal would be near impossible to federate. That said, if federation is your goal, MIMI+MLS seems to be the future. Matrix is moving towards MLS, Google's RCS encryption already uses MLS, and MIMI and MLS are often tied together in spec definitions. I believe the XMPP people are also working on (have finished work on?) embedding MLS in XMPP as an alternative to existing encryption methods.