We need to check ourselves on our personal heroes here. People are saying the same things about a specific American politician: "he's a good man, why are you attacking him?" If I were a philanthropist spending billions of dollars saving lives, would I get a free pass to kill a person a week? Of course not.
His opinion is not unpopular. It is morally reprehensible. Yes, some countries and states allow childhood intercourse, in much the same way that slavery was tolerated in many places around the world even after it was almost universally condemned.
Some minority voices are important (see: woman's suffrage, gay rights). Typically, these opinions are novel, they are new, they are revolutionary. Opinions such as the rights of pedophiles, or homosexual persecution are in contrast to hard-won battles that exist in the past. Regressive opinions are not plight, they are an appeal to return to the plight of others.
He is not supporting a fringe socially progressive movement. He is supporting something that we know is wrong. Pedophilia is an atypical anomaly. It is a debilitating disability. Those people need help desperately, not a pass to continue hurting children.
Edit: it also turns out that there are personal accounts of him being outwardly shitty to women. He needs a friend to course correct him, not praise or worship him.