Modern casual sexual encounters are assessed purely on subjective ratings of the opposite party's sexual attractiveness (secondary sexual characteristics, behavior, social status, etc.). Furthermore, constant saturation into the full range of these behaviors and characteristics across humanity through the internet and dating applications creates a endless "could I get a better mate?" feedback loop in the minds of bot…
I think your hypothesis is mostly correct, except for the idea that the men who are having more sex are “tremendous specimens”.
I don’t think they need to be especially attractive or powerful. It could be cultural or psychological differences that make some men put more energy into their pursuits. For example, compare Jimmy Carter to Bill Clinton.