Why? Well, for one thing these controls hardly work at all. Look at the population of illegal immigrants in the US to see how a long fence with vicious patrols and vigorous immigration enforcement still doesn't stop the immigration.
Secondly border and immigration controls violate human rights, without due justification. And no, "too much immigrants are inconvenient" is no proper justification for hunting, arresting, detaining and deporting actual human beings.
Thirdly cross-border trade and immigration is too valuable for both the poorer and the richer countries.