Clearly churches or religious institutions in many places serve the function of a public square, yet I can't walk into one and demand to debate anyone on the merits of atheism.
It's a very weird, sort of "in your face"-ism. Freedom of assembly means that said assembly can deny people entry or be exclusionary. That's just how a pluralistic society works. Also it's a very non-conservative idea, a sort of political gender quota