After living in California for a while and observing its urban planning culture more up close, I came away with the distinct impression that this is all by design.
California is owned by boomer homeowners, real estate speculators, and the auto industry, and the first commandment of all California planning is "allow nothing to change from the way it was in 1972." In the event that the first commandment fails, the second is "make anything new as expensive as possible so there's as much potential for graft as possible, and so existing home values aren't threatened."
Maybe things will change when there's a generational transition, but unfortunately it may just pass power on to the SV neoreactionary VC set who will proceed to implement their own different set of bizarre schemes that don't work like planned charter cities in the middle of nowhere. Or they might just turn into fuddy-duddies in their old age and change it to "allow nothing to change from the way it was in 2003 when I moved here and got rich."
California is the fourth largest economy, so clearly something is working, but I don't think California's urban planning culture is it. Places, companies, and people can all be extremely successful in spite of massive dysfunction.