Probably late posting here but going to post here to maybe offer some understanding to why it is people support policies like this. For the record, I'm not personally for it but there is a method to the madness. I'm black, take that for what it is.
"unacceptably white" is probably the most triggering thing in here but it actually represents the core of this conflict. The people trying to eliminate gifted programs are recognizing that many forms of power in society are zero sum. There are plenty of ways to get a great education but there is only one Harvard. Access to higher education opportunities is not only a pathway out of poverty but also to power within the institutions that hire grads.
The supporters of this form of affirmative action recognize that in a system where black/latino/indigenous students have structural disadvantages (poverty, prejudice, health, etc) but also operates as a pure meritocracy leads to a system where their racial group possesses disproportionately less power vs their population. Less kids in these groups in gifted programs means less of these kids going to top universities and on to top positions in government and business.
Many of you may see this as tribal or even perpetuating divides on race. But I'd respond that you fundamentally are underestimating how much marginalized communities distrust communities outside of their own. I'll use the black community because I'm a member of it. There is very little trust (myself included) that the white individuals who attend these programs and (hopefully) go on to succeed in higher education will use their influence for the benefit of blacks. I don't believe that because I think white people are racist, just self interested (like most people). That thinking extends to other groups like east and south Asians.
Personally my views on this are not to eliminate gifted programs, I benefited from one myself and know how important they are. Ideally we'd do what many others here suggested and just build more programs. If there is this much of a fight about it obviously there is a lot of demand for schools where Algebra I is taught in the 8th grade. So let's make more of those. That won't solve the issue at the college level though. There is a reason there are not 50 Harvards and its because the access/power/influence that degree offers is zero sum. Until we figure out a way to ration out that power in a way that people think is fair we're going to see constant contention over it.