That first of all depends what you mean by "diversity hiring". You assume it is not about meritocracy, but most diversity programs are exactly about trying to cut through biases and ensure a meritocracy. I see setting the two up against each other as largely a strawman - I don't know anyone pushing diversity programs that don't believe it is about evening the playing field rather than providing anyone with an advantage above and beyond their merits (I'm sure they do exist, but I haven't met any).
Consider this: Let's assume group A and group B both follows a bell curve in terms of abilities. Let's assume they're not equal. Maybe group B is shifted so that only 3% of group B can compete with the top 10% of group A, for example.
Now, if you hire exclusively from group A, and you due to competition for resources end up hiring from the top 20% of group A, you are losing out - there are then substantial numbers of candidates in group B that outperform people you are willing to hire from group A.
That is a non-emotional explanation for why diversity is good unless we're in a world where current distribution already accurately reflects merits:
Even if you assume that there are substantial differences in abilities between genders or races, unless you can consistently rule out that anyone in these groups are able to compete on abilities, creating an environment that is non-inclusive will leave your competitors with access to better people than you have.
The best way of ruling that out, if you believe these groups have different distributions of abilities, is to carefully control for biases and recruit on ability.
Now, if you mean quotas where metrics of abilities are ignored in favour of hiring people belonging to specific groups, then that changes the issue, though sometimes such quotas can also be justified if you have reason to believe the metrics used are flawed.
E.g. in the UK private schools tend to outperform state schools on A-level grades (equivalent, roughly to SATs / high school diplomas; used for university applications), but the conditions a top student in state school works under also makes it likely that their grades under-represents their learning abilities vs. private school students under equivalent conditions - differences in class sizes and available resources does make a difference in exam performance, and if you can correct for it, it makes sense to do so. If you don't, it may make sense to use quotas to ensure you safeguard your ability to skim the best candidates from these groups.
Incidentally, taking more students from state schools would on average be likely to take more black students, for example.
My (black, female) ex is on the diversity board of a major investment bank's offices in the UK, and is struggling with exactly this: The bank in question almost exclusively hires from a set of private schools where white men dominates, and sees "diversity" as being about attracting the few black people and women from those schools, rather than hiring by ability without looking at which schools the candidates are from, which would let in far more women and black people.
This is before taking into account other factors, such as positive PR with customer groups that hold increasing importance.
In other words, there are substantial cold, hard, emotionless reasons to push for less biased hiring, and the proponents of most diversity programs believes that these will provide more diversity through meritocracy, not despite it.