If you don't consider evidence you are aware of in an argument, it means you don't think that evidence is necessary to consider. He wants his argument to be considered rigorously in which case it is appropriate to make inferences about what he didn't consider as well as what he did.
It's a quiet truth that most disagreements between people have a crucial basis in the implicit nature of how we weigh and consider information.
A great deal of defense of Damore's arguments are to only consider what is explicitly said and not how it fits into the broader discussion, which is implicit by definition.
So Damore can claim he isn't saying that he thinks some female engineers currently at Google are there because the bar has been lowered (explicit) and this is accepted uncritically by some defenders, but it's not hard to draw the inference from his arguments and his policy suggestions that he thinks Google's current practices have a negative impact on the quality of engineers it recruits (implicit).
Likewise we are told to evaluate the information he gave (explicit) but not consider why he omitted discussing the mountains of evidence that suggest alternative positions and how his arguments should be evaluated in light of it (implicit).
My position is that there is the evidence of negative social feedback is significant enough that companies should experiment with changing their demographic compositions and empirically measuring performance.
Because software engineering requires a great diversity of skills, including people skills, empathy and whole-systems thinking, I think it's very plausible that making the gender balance less skewed will lead to more productive teams.