I agree with the thrust of your point - that organizations like YC should be allowed to uninvite or refuse to collaborate with people they are diametrically opposed to for political reasons.
However, I do not agree that it's fair to call an investor's competency into question due to their political views. I agree organizations have every right to not affiliate themselves with those investors, but in the real world there are plenty of examples of investors who hold controversial political views.
Logically, there is no a priori reason why "Thiel is a good investor." and "Thiel strongly endorses Trump." cannot both be true. To go further, I don't see an a priori reason why an investor with political views that are antagonistic to the industry he focuses on cannot be a very good investor. It might not be likely, but on its own I do not agree with using politics as a criterion for investing performance. That's silly - just look at investing performance directly.
To go even further, I think it would be disingenuous for groups like YC to announce they will distance themselves from investors they disagree with because "How could they be qualified investors if they hold this opinion?" That's masking the real reason for not affiliating with them, and in my opinion they shouldn't have to mask that anyway.
I also think using that sort of language establishes a precedent that makes the industry less inclusive to people with opposing political views. Much of the tech industry is progressive and liberal, but it would be ridiculous if this was applied to potential founders. Should founders who support SOPA or Trump not be admitted to YC if they are well into a startup that will most likely become the next Google?
Put another way, if you read Zero to One and you found it to be very insightful and full of legitimately good ideas, would you call it into question after learning that Thiel supports Trump? Doesn't that strike you as an ad hominem? Sure, Thiel's political views might be relevant or worth noting as an investor, but they certainly do not define his investment performance.