Even anecdotally, citing specific women who have done good work in this very specialized field, submitted a quality paper, and were rejected would go a long way towards corroborating the claim this conference is excluding women. Without knowing how many women submitted papers, and some (even subjective) indication of the quality of those papers, there is no way to judge whatever bias towards women this conference may have.
Also, are all identifying data stripped from the papers before consideration? If there is no way for the judges to even know whether the paper was submitted by a woman, how could they discriminate against her? If this is not the process, it may merit consideration for future conferences.