Yeah, this gets into a problem where people use the word "sexism" in a couple different ways, that don't mean the same thing. Mostly conscious vs unconscious, I think? Almost no one goes home and writes, "Dear Diary, today I insulted some women, because I hate women. Let's keep doing that." So of course, if you accuse someone of being that kind of conscious sexist, it's a huge insult. You're calling them a sad hate monster person. When we do talk about other people's sexism, especially if those people don't have a lot of practice talking about unconscious bias, it might be a good idea to reassure them that we're not calling them a hate monster.
But that cuts both ways, and it's possible to be unfair in the other direction. If someone comes to me and says "I think what you did was sexist," it would be fair and polite of me to assume they meant "unconsciously sexist." If I want to ask for reassurance about that, sure that's fine, but not fair for me to put words in their mouth like "HOW DARE YOU CALL ME A CONSCIOUS SEXIST?!" when I know that both meanings exist.
So yeah, sorry for the long reply, but here's why your comment made me think of it: Saying "that wasn't sexist; so-and-so is actually less assertive" kind of assumes that we're talking about conscious sexism. If we're talking about unconscious sexism instead (the charitable assumption here), that difference-in-perceived-assertiveness might be precisely the problem that we're worried about. This is kind of a nitpicky distinction, except that accusing someone of accusing you of conscious sexism, can be kinda sorta flirting with bad faith?