I think this article greatly misrepresents the main arguments going around the Fediverse about Signal, and the arguments for alternatives.
1. Signal holds the only set of signing keys to the only published binaries allowed on the network, compiled by them, that in turn control all signing keys. If pressured they could push a malicious update with unpublished code. The published server code has not been updated since April last year, so either they have made no changes since then or they are already comfortable pushing updates without matching public source code.
2. The bulk of metadata protection on Signal comes down to trust in SGX, which indeed is an entirely broken technology and keys can be extracted from it via a number of side channels if there is sufficient motivation, such as a government trying to track down dissenting views, or a future owner of Signal that secretly is willing to cooperate with a state actor. Intel also could, if ordered, also issue a microcode update intentionally compromising the RNG used for keys, etc. Signal places a huge centralized target on its back so I think these risks are plausible and worth being aware of.
3. Signal forces all TCP/IP metadata to one stack, which if combined with heuristical analysis, I strongly suspect it would be possible to work out which IPs communicate with which other IPs even without aid of SGX contained metadata.
4. Signal is actively hostile to any third parties that compile and sign signal network compatible binaries and release them via open source app stores, and vows to fight them and get them removed from the network. Moxie repeatedly says he prefers the install base tracking proprietary stores like Google and Apple afford.
5. Signal has built their entire social graph on phone numbers which require ID to buy and are actively tracked in 200 countries, and many carriers will sell out their customers to bounty hunters etc. This is directly at odds with their stated goals of furthering privacy.
When asked what alternative I suggest, I say Matrix.
Those that really need privacy can use a pseudonym via Tor on a server of their choice hosted by people they trust and avoid revealing PII to the messenger at all, unlike Signal. The most private metadata is that which you are not required to reveal in the first place.
The server and remaining metadata that must exist, like sensitive channel memberships, easily be hosted in a server you own on property you own. Sensitive channels could stay on that server and not reveal any metadata or group participation to the wider network giving you granular control of your own metadata and where it lives.
If you really wanted to you could even use generic tools that exist for SGX to do FDE on the database disk with the key in SGX and in turn also run a lean binary like Dendrite in SGX. I don't think this is worth it, and I think SGX is largely security theatre at this point but this is what freedom looks like.
You can run your own server and maintain it according to your own threat profile, instead of using a one size-fits-all threat profile a centralized walled garden forces on you.
If you still think all the arguments above are totally unreasonable and you don't like hearing a lot of opinions critical of popular centralized services like Telegram, Signal, and Whatsapp... then indeed the Fediverse may not be for you.
Most users care a lot about keeping digital sovereignty which is why they joined the Fediverse in the first place.