Because growth isn't even nor can it be, it depends on connectivity of an area to relevant features that spur further growth, such as industry, jobs, education. There are highly specific reasons for growth to happen where it happens, and 9/10 it's due to some piece of immovable necessary infrastructure that supports a wider region. For example, Chicago is the size it is because it is a confluence point of freight, first from the Mississippi and the Great Lakes, now freight rail. Los Angeles is the size it is because the Port of Los Angeles/Port of Long Beach is the mouth of all east Asian trade coming into the continent, which started a century of industrialization and job growth in the area. San Francisco and Boston are innovation centers due to the number of high quality universities in these areas that continually feed the local talent pool by the year. There are plenty of places that are zoned for growth but it hasn't happened yet, either. For instance, just across the Angeles National Forest north of Los Angeles, you have the Antelope Valley, which remains relatively depopulated in comparison to Los Angeles due to a lack of connectivity to very many job centers or much industry, even with roads having long been laid for eventual development (1).
1. https://www.google.com/maps/@34.5617689,-117.9209187,5165m/d...