Earlier quoted context omitted.
> This problem can only be solved politically, with the state taking a major role in the supply of housing The state already takes a major role in the supply of housing, as you yourself acknowledge. That's the problem, not the solution.
The upper levels of government can and should correct the failures of the lower levels which are captive to local homeowner interests. The ideea that you can eject the state from regulating the use of a resource with so many externalities is a pipe dream, you either get some sort of favela or the residents re-invent a much more exclusionary and bigoted version of the state, for example the HOA.
They have been, for decades. That's where things like the Community Reinvestment Act, which played a significant role in causing the crash of 2008, came from. Not to mention the decades-old policy of encouraging mortgages by funding them with money printed by the Fed's printing press.
> The ideea that you can eject the state from regulating the use of a resource with so many externalities is a pipe dream
No, what is a pipe dream is the idea that governments can somehow fix problems with externalities or market failures. They can't; they make them worse. That's because governments are not magical entities that somehow always do what is best. They are organizations run by human beings, and their incentive structure is even worse than that of free markets.
> the residents re-invent a much more exclusionary and bigoted version of the state, for example the HOA
While I'm no fan of HOAs, at least the people in them have skin in the game, being members of the community themselves. However flawed this model is, it's still preferable to having unelected bureaucrats in a city a thousand miles away micromanaging people's lives.