Then you don't understand the core problem.
You know what, I think people, in general, should be allowed to discriminate. The whole "right to refuse service" shtick - people shouldn't be forced to do things they don't want to do, even if they are following stupid beliefs.
That said, I recognise there are things that have to beat that. Just like free speech doesn't include shouting "fire" in a crowded theatre, we have to curtail people's right to discriminate where other people's lives are made unlivable because of it.
If someone refuses me, the white straight male, the chance to buy something, I go somewhere else. That's not the case for some minorities however. If everyone won't provide wedding services for you, or everyone won't let you attend a class, or everyone won't sell you food, you can't live your life. Commonly discriminated minorities have to be protected because they have a right to do what they want as well. Just as you are free to do what you want, but that doesn't include murdering people. You are free to refuse service, but not based on things that mean that person can't get service anywhere.
It's not perfect, but it's a solution.
Now, of course, that doesn't address another salient question - even if it's legal, why would minorities want restricted events?
Well, culture. Obviously in an ideal world events would be open to all and it'd all just work. The reality is, however, that culture can be a barrier. Let's take chess as an example.
Chess was male-dominated in part due to lack of education for women and it being seen as a male game. Obviously tournaments today are not restricted to men, but we still see less women.
That is, at least in part, because women never get exposed to chess because there is still cultural pressure that it's a "men's game". Women are excluded when they are young and never get a start. The intent of girl's only clubs and tournaments is to encourage women to get into chess, then they integrate into the larger chess scene. As time goes on, the societal stigma lessens and hopefully we end up in that ideal situation.