I think Apple's approach is the only reasonable one for the general population. The technological complexity of any smartphone is far beyond comprehension for most people.
I write iOS software for a living, and even with complete access to the source code, I couldn't reasonably evaluate my iPhone's software - let alone the hardware.
The idea that ROMs from questionable sources make your device safer sounds very strange to me.
Basically every electronic device has countless security issues. Some of them are found of which some are published of which most are eventually fixed (by rather large teams of professionals). In that regard, Apple could and should do better.
But the burden of making such a complex device secure simply can't be put on the end user.
While I would welcome deeper access for technically inclined people, I'm not sure that option can really be given by Apple/Google without the risk of becoming a disadvantage for many users.