And that sucks. We need you to share your experiences. We need to feel your struggles and understand them well enough to identify with them, and really internalize, "what if that were me?"
When that happens, most of us respond with empathy and because we understand, support and activism to make it better and have one another's backs.
Progress happens, and things get better. I believe in all of that deeply. I've paid hard to stand for it too. More than once.
Some of us respond with rage, hate, and all manner of emotions, and that sucks. It's not like any of us actually picks these things, so how does it make any real sense to be held to account for them? (Color, gender, gay, etc...)
It is, like you say, a contact sport with very real implications for us all. To me, that contact is good. It helps to understand others and it can help to keep the worst in check for fear of consequences too.
But there is a lot of difficulty in that quiet response one just can't quite pin down... fired one day, for example.
I'm an ordinary white guy, and I've had this sort of thing happen when I've gone to bat for someone who needed it.
Making things better is expensive in this way, and I'm not entirely sure we can avoid it. We may blunt it some, but people remain exposed all the same.
I understand your fear. It has a real basis and all of us must weigh these things. At some points in life, maybe we can take the hit. At others, it may cost too much.
I don't have answers, other than to say there are a lot of us who get it.