Earlier quoted context omitted.
visa restrictions into the US and EU exist to protect their own population, as voted and decided upon by their democratic systems. the world is a bit more complicated than your comment implies.
This is a clear example of tyrrany of the majority. The majority of people might not want to move state, which is fine, but they prevent other people from exercising their right to freedom of movement. But this ends up being bad for everyone - if we are agreed that competition is good, this prevents competition between sovereign states for labour and population. As an individual, I am stuck with whatever citizenship…
If you think that a country giving privileges to its own citizens compared to foreigners is morally equivalent to a caste system, you might as well claim the same about parents caring more about their own children than other people's children. Good luck with that.