Embracing compassion for all those who drift into our orbit is the only way to improve oneself in all dimensions. Compassion is the common factor in virtue, and the cure for selfishness.
It is also the measure of every person, so once you begin orienting yourself in the moral compass' proper direction, you can begin to see how others are aligned or misaligned. Such selfish folks are those who just don't give a damn to become a better person, which would result in lessening the amount of hurt they inflict on others. Treat such people as well as you can, but be wary of their nature.
Unfortunately, very few human beings really give a damn about anyone not in their in-group. Such is our baseline mammalian heritage, which we must each overcome with deliberate spiritual self-evolution.
"The Way goes in." --Rumi