Planning is a perverse set of incentives. First it’s usually longer term thinking, where decisions made now impact the shape of a city decades out. Second, the people that live in a place now have undue influence and veer conservative in how land is used (ie NIMBYism). This often gives rise to at least near term stagnation (on a historical scale) in established, wealthy cities and growth in up and coming metros that…
Among the most beautiful cities in world are those that were not planned, but were just allowed to happen. I don't know where we get the mindset that planning is necessary to begin with
Also, we now have the automobile to content with. The car is a like a gene drive, a self-perpetuating technology like we had never seen before, radically upending status-quo-ante system dynamics. (The reasons being a) Cars push everyone else off the streets, b) car-driven development spreads everything way to far apart for anything else to be practical.)
The market alone can't defeat cars any more than evolution can defeat gene drives. Only social failure / extension (and heaven forbid horizontal gene transfer!) can rain them in "endogenously".