It's a simple problem, we turned housing into an investment. Now everyone who owns a home is incentivized to ensure their investment is maximized and that results in a lot of NIMBYism. The simple solution is we need more housing in cities and suburbs. How we get there is an exercise that's taken on by local housing authorities and they are not incentivized to solve the problem. That's because they work for voters who…
I don't know how this came to be such an obviously true common wisdom answer here at Hacker news. It's an extremely incomplete account of what causes NIMBYism. I'm sure at least some of it is purely a desire to maintain land value appreciation, but as others have already pointed out, in plenty of cases, conversion from single-family to multi-family zoning increases land value and the land owners still don't want that.
The basic cause of NIMBYism is these are people's homes you're talking about. They live there. They may have lived there for decades, possibly even for generations. They chose the place for the character and culture it had at the time of choosing, and they don't want that to change. If they enjoy a relatively quiet life with space, they don't want to see their home changed into a traffic jam with nowhere to park. They don't want to be perpetually surrounded by nonstop construction.
I'm not trying to say anyone who feels this way is inherently right. At some point, change is inevitable. I'm sure the original inhabitants didn't like it when European colonists spread strange diseases across the land and then wiped out the stragglers with superior firepower. I'm sure the ranchers of the west didn't like seeing their ranches turned into suburbs. And the suburbanites don't want to see multi-family housing in their once-quiet and relatively secluded homesteads.
Without making value judgments either way, though, I think you have to acknowledge there are far more basic instincts at play here than simple financial incentives. It's fair to say we at some point have no choice, and if the result is we can solve housing unavailability and especially homelessness, and the cost is a few or even a large number of people are left with the neighborhoods they grew up in changed drastically into places they no longer want to live, that may be worth it. But you should understand the full range of motivations for people who want stability in their homes and communities and not act like it's purely about financial gain and greed. Remember also that a fair number of communities in which concerns about gentrification are the greatest and people really oppose redevelopment are neighborhoods that these people were historically forced to live in because they were ostracized if not killed when they tried to live anywhere else. That community in its specific current form is what they see themselves as being a part of, far more than the larger city, state, or country that never wanted them in the first place. This is far more complex than you're making it sound.