Because the systems which identify students and enable them to meet the admissions requirements are massively unequal.
I'll give an example. I attended a gifted program starting in 3rd grade. I went to an elementary school for 1st and 2nd grade that was almost entirely white with some asian kids. Part of the mechanism for selecting students for the gifted program was a letter from the student's counselor, teacher, and principal.
My mom noticed that some schools were sending way more kids to the GT program than others. Several schools basically never sent any kids. No surprise, there were more black and hispanic kids at the schools. BUT these schools did not have a lack of students with high SOL scores or grades. It turned out that the process of identifying students and writing letters was letting these kids down. A few years of local activism to adjust the process and suddenly a bunch of kids from these other schools were being admitted to the GT program and succeeding.
Activists recently tried to adjust the admissions criteria for TJHSST, a magnet school in the area, to be a "merit lottery." This says that you set some minimum qualification (minimum GPA and accelerated math coursework) and then select students at random from that pool. This proposal was called racist against south and east asian students and activists were called enemies of excellence by various people like Scott Aaronson.
I was personally identified as having potential for going to TJ at age four. My parents paid a lot of money for me to have tutoring specifically for the admissions test.
In the above example, black and hispanic students with similar qualifications (test scores) are not receiving the same access to accelerated programs.
The point is that the mechanisms by which we define qualifications and select students really really matters and can encode all sorts of inequities.