Earlier quoted context omitted.
The problem is what happens when economic migration intersects with modern democracy and welfare states. Because of the state, migrants (who should be a net positive due to comparative advantage) suddenly become an economic drain and can harm the people in the society they enter. This happens both by taking resources from workers via the welfare state and by voting for harmful policies (though this last part usually…
That's been done in late 19th-century America, and the result was mass exploitation of immigrants and increased violence. In fact, regulation in the US (unlike in Europe) started as a response to what happened then (the US population cried for more regulation to save them from the Robber Barons). Of course, it was also a time of great economic growth but also third-world-like inequality. So what you're suggesting isn…
I think that most white people in the south also preferred the current, perhaps more subdued reality of segregation to the constant struggles of integration and economic competition with "those people". Is that a valid argument against eliminating mandatory discrimination?