"Google Play Services lets Google do silent background updates on apps on your phone and give them any permission they want. Having Google Play Services on your phone means your phone is not secure." Yes, Google can install a backdoored version of Signal. This is bad. But if you can't take that risk, you can install e.g. LineageOS without Google Apps, download the source code, reproducibly compile the apk, and instal…
All your suggestion does is, it adds a layer or two where we hope the NSA doesn't compromise them in case you'd want to use that chain to install and validate Signal.
You can't possibly think that compromising the full infrastructure of MIT or F-Droid, a noisy criminal act with serious repercussions against the perpetrators, is in any way comparable to a MITM against a suspect.
That's like saying "Ok, North Korea has some small nukes, but if they really want to get serious about a nuclear attack, they can always penetrate the White House and steal the nuclear football from under Trump's ass".