Zoning is the issue in some cities, but sometimes it isn't the root problem.
Robert Caro gets into this a little bit in his book "The Power Broker" describing Robert Moses' development of highways in NYC: basically, if you build a highway before you build a subway line, the resulting area will be developed in a low-density pattern (regardless of zoning), as the "last mile" of driving is much less difficult than the "last mile" on foot, coming off a train.
If you look at NYC in particular, this happened in eastern Queens -- the highways were built (with poor public transit coverage) and the properties were developed in a low density pattern.
This cannot be fixed by simply building more high-density housing (despite the massive city-wide demand for housing), because no transit exists. But you cannot build new transit because it's a political nightmare: current residents will complain that you're disrupting their neighborhood, and city residents elsewhere will complain that your new transit development doesn't "go anywhere useful."
From what I saw in LA when I visited a few weeks ago, much of the new systems of transit serve (a) large office campuses (I was regularly taking the metro at Universal City) and (b) non-commuter destinations such as Hollywood (for tourism) and the airport. The result, as pointed out to me by an Uber driver, was a huge explosion of development in Downtown LA. The newest line in LA, the Expo Line to Santa Monica, seems more commuter focused -- perhaps it is serving the new residential population moving into DTLA?
Eastern Queens, unfortunately, doesn't have the luxury of important destinations or office campuses. It's overshadowed by Manhattan in terms of tourism, and Moses built the Long Island-bound highways specifically to prevent the sorts of development that would lead to Long Island becoming a commuter destination rather than a residential community. Thus, Queens is trapped in this residential catch 22, and not even rezoning can fix it. The only solutions I can see involve closing down lanes of highway to obtain right-of-way for transit (the overpasses were intentionally built too low to support bus service, so it'd be a permanent closure), a sure path to political suicide in an area utterly dependent on its highways.