Where we disagree is probably in using the language of "privilege" and related concepts. I see no point to it, and it's probably harmful in the long run. It's imprecise (typically dealing only with vague averages), ignores detriments, ignores privileges when they imply the existence of detriments (i.e. there is certainly female privilege, but it is often ignored because that implies that some male privilege could be counter-balanced to some extent by some male detriment, i.e. not being a female), applies poorly to individuals (because they are so multi-dimensional), ignores complicated combinations in favor of simple ones that seem only to infer positive correlation (white cis male etc...), and is championed most loudly by means of pathos appeals. Why bother with that framework?
I don't think we should stop talking about gender, or class, or race, but we ought to be more careful about it, especially when we try to assign causes, and especially when we talk about individuals. Focus more on ethos and logos. Use full data sets, talk about positive and negative, talk about interference both constructive and destructive. Make predictions and update when your predictions are falsified. Make predictions about data before you look at the data to see if your priors are well-calibrated. Talk about consequences more than feelings.
For the last point, it's mostly a complaint about using a (mean) average for everything without considering whether the underlying distribution makes such a statistic worthwhile to use or not. Consider a block of houses valued roughly the same amount, then some rich dude comes in, knocks one down, builds a towering mega-house that hits the limits of what is legally allowed, and then sells it to some other rich dude. His house has bumped the (mean) average house value of the neighborhood up considerably, but no house in the neighborhood is actually worth the average. Talking about the (mean) average is then pretty useless, and no one is (literally) going to buy into one of the normal houses being worth the average of the neighborhood. The median average is much more useful, at least for property values. But even then you'll want to look at the full distribution to say whether the median is really more useful or not, and when you're buying or selling a specific individual house, you're going to want even more details about that specific house that could drastically change the valuation one way or another.