Here we run up against the fact that the world is a far more cruel place than most people would like to admit. Is it fair to these people? No. Do I like it? No. Is it going to happen anyway? Probably.
It's worth remembering the history of people who love their home but had to move away because some other people loved it more. Those folks who are so attached to the downtown apartments they can't afford anymore? Their neighborhoods became desirable because of a variety of urban renewal projects that largely worked by eminent-domaining & bulldozing lots of poor, mostly black neighborhoods. The original inhabitants - who after the Civil Rights movement were actually excited that they might be treated as first-class citizens - had their communities eviscerated, and still haven't recovered.
And then if you go back further, all of America is founded on the genocide and conquest of the Native Americans, many of whom loved their home and had to move away because the location was highly desirable for settlers.
And if you go back further, many of those tribes themselves stole territory from other more peaceful tribes. The Comanche and Aztecs, for example, practiced slavery and human sacrifice for rival tribes captured in warfare.
You could look at this as part of a long chain of humans kicking out other humans whose lands they covet. In the grand scheme of things, it's actually rather polite, as the original owners are being paid for the use of their properties and aren't even giving them up.