Good chance that this long-winded response will end up being lost in a stale thread, but in case you happen to come back to it:
I suspect if we get down to concrete descriptions and prescriptions, we'd have a lot more agreement than disagreement. So maybe all of this is semantic, and a bit far afield of Scott's original post.
Privilege is an important concept: many people, however, elevate it to a metaphysical reality, instead of a set of cultural practices and institutions situated in a particular social context. The idea seems to be there's white and black, male and female, rich and poor, and that set of binary relations is defined by membership in a hardcoded hierarchy of oppression. Even "intersectionality," which most everyone is already familiar with, in practice seems to be mostly about "there are lots of checkboxes you can check, and you'll always have some that are checked and some that aren't!" More sophisticated takes on it acknowledge the incommensurability of experience and use that to condemn oppression olympics[0], but it falls prey to a core problem.
Privilege ends up being a very leaky abstraction. "White privilege" (privilege itself, not the term) is a modern invention: white privilege didn't exist in 1500, because white people and black people didn't exist in 1500. Race was constructed in the early modern era to provide the ideological superstructure for the pattern of colonialism and domination originating from the western coast of the Eurasian landmass. And the strange fruits of its maturation in the late 19th century remain with us to this very day, to the extent that in the vast majority of contexts, you're better off being white than black. So I'm very comfortable with using a monolithic white privilege as an explanatory concept when talking strictly about black-white race relations.
Even that, though, breaks down when you consider the multiplicity of races: you no longer can have a simple binary of the oppressed and the oppressors, because you have to account for East Asian, Native American, South Asian, etc. experiences as well. You can try to bundle all of them together and talk about "white privilege over nonwhites," but that's pretty problematic itself, because it reduces all non-white racial experience to simply being the Other. Or you can come up with a whole set of racial binaries--white and Asian, white and black, white and non-white Hispanic--which respects the unique discourse that defines each race, but runs into issues when you start talking about, for example, black-Asian relations. I don't think this at all undermines the usefulness of white privilege as a concept, but it's a very subtle concept.
The point of all of this prelude? Even in the simple case of one clearly dominant group, treating white privilege as a monolith is tricky, though I think ultimately it's doable in a way that's useful and powerful. But that brings me to my main point: male-female privilege, which nowadays is better characterized as male privileges and female privileges, though in the past it was very one-sided. The current discourse of privilege, however, is unable to account for that complicated reality, because it insists on parts of each individual's identity being classed into a binary of oppressed and oppressors. And because of that, it very much discredits people's real struggles. It categorically rejects the possibility that men suffer systemic oppression based on their maleness. Men who fall outside the male archetype, in body, personality, or how they are socially situated, are marginalized and face economic and social discrimination. And even men who manage to fall within the narrow scope of our society's hegemonic masculinity suffer a great deal due to it, resulting in harrowing statistics like men comprising 3/4 of successful suicides.
At worst, the response to this from activists is nastiness, mockery, and derision ("I drink male tears!"). Sometimes the response is saying that that sucks, but it's not systemic oppression so it's not worth worrying about, or maybe that men committing the large majority of suicides is bad but it's different in scale from the oppression every woman experiences. Best is the more empathetic and sophisticated ones who cite "toxic masculinity", which is fine as far as it goes: it acknowledges that men suffer based on their gender, and I can enthusiastically be allies with people who recognize that. But it strikes me as trying to fit the square peg of oppressive male gender roles into the round hole of privilege. For point of comparison, consider how ridiculous something like "toxic whiteness" or "toxic affluenza" sounds. But otherwise there's no way for privilege discourse to acknowledge that there's anything at all that sucks about being expected to perform masculinity.
So, that ended up being quite the essay. But thanks if you read it!
[0] Though, isn't "acknowledging the distress while continuing to point out the difference in scale" really just oppression olympics under a different name? Or does this come from the point of view that oppression olympics is a valid intellectual exercise, but isn't useful tactically while collaborating with diverse sets of people?