You say the "core element" is that one shouldn't feel ashamed to be privileged. Well, fine: one also shouldn't feel ashamed of being a witch! And why would one?
If I were a witch, I wouldn't be ashamed of it. I'd be proud of being a witch. There's nothing inherently wrong with being a witch - witches are cool. The fact remains that the claim "if you deny being an X, that REINFORCES X-ness and lets X feed upon itself" is toxic and ridiculous. And the main other assertion I could think of that had this bizarre characteristic besides privilege was "you're a witch!" (I suppose "you're a racist" also has that attribute, but I didn't want to go there. Can you think of others?) "Witch" accusations are great for our purposes because it really has no moral characteristic at all. We all know witches don't exist and can't exist and I assume we don't collectively subscribe to religions that are afraid of them (or even if we do, we know that that fear is silly), so it's safe to use them as an example.
So. With regard to any particular person, if they say "I'm not a witch" that MIGHT just mean they're not a witch or they don't regard themselves as one. And that's OKAY. Even if the claim is MISTAKEN and they actually ARE a witch, it's not the case that denying it is uniquely evil. When witch-hunters claim it is, they're wrong. Any person denying that THEY are X doesn't really say anything about whether X COULD exist generally.
> There's also something poetic since claims of "witchery" are a centuries-old tool used to silence women.
Similarly, claims of "privilege" appear to be a modern tool used to silence men. The core essence of privilege seems to be that you have it when a bunch of people (of the right sort) SAY you have it (whether you agree or not) and it's considered rude/offensive/impossible to deny it.
You said:
> I'll try to illustrate why privilege is not a scarlet letter but denying it can make one complicit.
...but never did anything to illustrate why denying it "can make one complicit" - you merely asserted this. So I assert the contrary: there is nothing wrong with simply being colorblind, gender-blind, every-other-characteristic-blind and treating people as HUMAN BEINGS first and foremost. Overemphasizing race/gender/handicap is at least as big a potential problem as underemphasizing it. If you always bend over backwards to make sure Tara gets asked what you feel are the appropriate number of questions given her gender, that is patronizing to her. Simply treating her as an equal would be more respectful and productive. Why would you need awareness of "privilege" to pass questioners to the person in the room who is the most relevant expert?
Nobody denies that some people have more advantages than other people. The only thing we're denying is that having some random set of advantages invalidates your opinion. The difference between privilege (what we're arguing about) and advantage (what you just gave examples of) is that privilege has the immediate additional connotation "and therefore you should shut up because it's wrong for you to express views on this matter". If it didn't, we wouldn't be having this discussion.
BTW, Louis CK's routine is funny but he's wrong. (Which is part of what makes it funny.)