This is a complicated question, and for those new to this kind of discussion, it deserves a complicated answer. This is not that answer, but here are some thoughts:
a) We are a reactionary species, unfortunately. And that served us well when the burning questions of the day were "Tiger friend? [BITE] No, tiger never friend" but less so when discussing the social weight felt by minority and historically persecuted groups, the benefits and pitfalls of hyper-vocal "allies" and the social disenfranchisement felt by potential allies bitten by persecutions that they are told do not exist.
b) We are a reductionist species, unfortunately. That helps us when understanding the complexities of mathematical infinities and physics and the workings of the human central nervous system, because - between similarly educated peers - a scientific community can abstract away the irrelevent bits and share groundbreaking research outcomes with relative ease. But when "normals" try to enter the discussion, they mistake the simplified abstract descriptions for a simple concept and end up contributing nonesense or reporting falsehoods to the rest of the world (e.g. are we moments away from fusion generators? Are we on the cusp of ignition? No, no we're not).
And so, when discussions take place about the relative protections and privileges inherent with social majority groups and justify the particular resource expenditure on helping to rebalance centuries of systemic prejudices, people will misread what they understand of the zeitgeist and say "Your social privilege, as defined by your [race / gender / sexuality / ability / age / etc] means that your feelings need no consideration"
c) We are a self-absorbed species, unfortunately. It contributes to our survival and that of our dependants to ensure that we take the resources we need so that we are in a position to then help the rest of our clan. Weirdly though, we also have faculties for patience and imagination. And so we are happy to imagine ourselves in the shoes of someone we consider as "sufficiently other" to the point where we will fight for them as if they are our own clan, which is in a way wonderful (not always - see "white saviourism") but it means that those similar enough to be relatable receive a negative consideration, being not different enough to be worth the emotional investment, but not similar enough that they are truly "in the same boat", though we feel like they are. For example, try to analyse your perceptions of the poor and homeless that live near you, compared to the poor and starving people from distant countries.
It just "feels different", right?
That "local mistrust" is just as much an undeserved prejudice as the fact that you probably imagined those "people from a distant country" with a particular skin colour.
And lastly:
d) Our emotions deceive us constantly. That small racism dig in part C - if that affected you (sorry!) might make you susceptible to further emotional manipulation, because (see A) we are a reactionary species, and so from there it's easy for someone to manipulate us to reduce (see B) the world into "overprivileged vs underprivileged" clans, take on the underprivileged as our own (see C) and thereby increase our prejudice under the guise of eliminating prejudice.
Then echo chambers, then reinforcement by media, friends, catchy superficial soundbites, our employment... and finally our own selves, because once you've declared your unwavering support for a vaguely positive cause, it's very hard to walk that back.
And convincing someone to reverse a commitment they have made that has superficially positive soundbites because they are taking a damaging approach is a lot harder than reversing a prejudice they fell into through normal upbringing.
Maybe, long-term, that isn't a problem. Over the generations, we'll react ourselves to a position where everyone is protected. Short-term though, we'll have to use our privilege to be a force for good in between protected factions.