I get that, and sort of sympathized at one point. But for one thing, if you're not allowed to avoid the controversy except by not speaking at all, then the hands been overplayed.
Personally, I always try to draw the line from a natural rights approach. If one "right" requires the infringement of the rights of someone else, then it's not really a natural right. If your personal belief requires me to have a certain personal belief, it's not a natural-right, it's a form of religious jihad.
People need to realize that if you aren't "normal" in some way (whether it's a personality quirk, weird hobby, odd fetish, or disability, or whatever) that life isn't guaranteed to be easy, and you're going to need to take certain conversations or commentary with a thick skin. (I know "normal" can be a no-no in this topic, but I mean in general terms, which I think is still appropriate for this context.)
And the "coerced" or "forced speech" approach really doesn't make sense in practice if we're strictly looking at self-identity. If it were strictly about self-identity, we'd still say pro-life and pro-choice, instead of pro-woman and anti-abortion. "Racist" would basically be off limits except for the very small number of people that openly embrace it as a badge. Making any assumption about anyone's outward appearance (or any assumption about the underlying motive of anyone's actions) could land you in trouble with the thought police.
People say 1984 and BNW, but I think the whole thing is...it's very Victorian- just with a different set of taboos and standards. Although with the direction me-too is headed, with people being offended (or even feeling harassed or assaulted) just by being asked-out in (what they perceive to be) an offensive manor, we've almost come full circle.