It's fair enough to characterize King's work as part religious ministry, and I think it's also fair to consider what it means that King's walk didn't live up to all of the standards one would assume would come with his apparent Christianity.
I'm not sure it is in fact fair "to use those moral standards against him", though, unless you're working with a conception of moral standards as a game in which the point is to cast your team into The Good Guys and the other team as The Bad Guys. And sure, some people play that exact game (the FBI is doing it here), but you can also approach moral standards as ideals which would make the world better if we could adhere to them. The latter conception still means that people who fail to keep standards can suffer natural consequences (as well as artificial ones of standing) if they don't adhere, but it's not much of an attack on the moral authority of the standard.
There's also a question of a sort of standards severability. King is known for agitating for racial equality and social justice, not for being a crusader for the virtues of chastity and fidelity. If he'd been known to privately abuse and privilege based on apparent race, or inclined to acquire wealth at the expense of others, that'd seem be a bigger deal.
Finally, a little bit of C.S. Lewis:
"The sins of the flesh are bad, but they are the least bad of all sins. All the worst pleasures are purely spiritual: the pleasure of putting other people in the wrong, of bossing and patronising and spoiling sport, and back-biting, the pleasures of power, of hatred. For there are two things inside me, competing with the human self which I must try to become. They are the Animal self, and the Diabolical self. The Diabolical self is the worse of the two. That is why a cold, self-righteous prig who goes regularly to church may be far nearer to hell than a prostitute. But, of course, it is better to be neither."
Again, none of this is to discount the value it might have to consider an MLK with weaknesses and some questionable behavior. Or, for that matter, the value of fidelity.