That's a reasonable argument; we should think it through to the conclusion. In a more general form the argument is:
Given that certain forms of discrimination are unacceptable, why is okay for any group to discriminate against any other group? Why is it ok for a nation to enforce borders, a company to keep the proceeds of a new idea to themselves, or for a wealthy family to favor their own children?
Many people have asked that and tried to create systems that forbid such discrimination. Unfortunately the result is always disaster. It's simply so counter to human nature that it cannot be imposed without a totalitarian government and the problems that come with totalitarianism are far worse than the problems of inequality that come with freedom.
But perhaps a moral person should still seek to follow such a policy of their own volition? That might be viable, if dealing only with other people who follow a similar policy. But how many people are willing to work for such a system, rather than merely using "equality" as an argument only when it serves their own self interests? How many are willing to make the sacrifice when it's their turn to do the dirty job, to work hard without keeping the proceeds, to share their home with strangers, to fight criminals and tyrants with nothing to gain for themselves?
That's an empirical question and many experiments have been conducted on communes, kibbutzim, ashrams, etc. Experience has shown that the answer is: far too few to make such a system viable. Even voluntary societies of this type are plagued by freeloaders and cliques, and must impose borders to prevent being overrun by outsiders.
So while it certainly seems like a good moral position, and has seemed so to many, many people throughout history, it simply fails to work in practice, over and over and over again.