I understand. Here's the funny thing, though: most people think the "cheated on" partner isn't aware of the "cheater" doing what they do.
In a lot of cases, the arrangement is tacit and implicit. They simply don't talk about it. Call it denial if you will, the same denial a heavy drinker getting home smelling alcohol doesn't trigger the "Have you been drinking?" question from their partner or family member. Now, the drinker might truly think the others don't know, or he might be aware that they know but simply choose not to address the issue. So nobody talks about it.
And then there's the explicit arrangement where the one who's "cheated on" tells the one going around that the only think that would be embarrassing would be people knowing he's cheating on her. In other words: Don't put me in the cuckold position in the eyes of society. It's not the thing that's the issue, it's the PR of the thing that is.
This is the same thing as Kevin Costner in 13 days telling the pilot not to get shot, because if he is shot, the President must do something because the people would all look at him and expect him to do something.
In other words, what the cheated on is looking for is plausible deniability. "I don't care what you do as long as I'm not in the position of people waiting for me to act". If they don't, they'll be the parter tolerating their husband/wife cheating, and they'll be shamed, ridiculed, etc.
The attacker is well aware of these dynamics when he exploits this to destabilize his opponent. He's exploiting and using the cheated on as a means to an end. Again, my point being that we give these people an audience to make the argument they present pertinent ones.
As an example, it used to be that someone who was "exposed" as gay would see his career ended. He'd be done. It is much harder today. There's still much to do because it's not all pretty and equal in the real world, but I don't hear any politician saying "Well let me tell you a secret about Mr. Smith.. He is a sodomite!" and the audience going "Oooh!".
It worked when the audience went "Oooh!". It works less well when the audience says "Meh.".
An argument is only as powerful as its pertinence to the target audience. We are the problem.