"The cause of the crisis is the array of regulations against homebuilding, that caused the shortage."No. That's what YIMBY people keep repeating. Regulations account for only ~20% of the cost of new construction in SF. The "cause of the crisis" in SF is acute demand on a tiny peninsula where there is no unbuilt land. I can't speak for the rest of the bay area, but the YIMBY line is almost entirely wrong when it comes to San Francisco.
"In a normal situation, increasing prosperity would bring both jobs and homes to an area. Here, we have the jobs but not the homes."
San Francisco has been in the midst of a dramatic building boom since ~2013. There are projects all across the city. Mission bay went from vacant land to fully developed in less than four years.
In other words, aside from the fact that huge multi-family housing projects take a few years to get built (everywhere!), things are working exactly as they should. But no, you can't build giant housing projects faster than the tech bubble has inflated in SF. That part is true.
"The problem here is that California has some of the most misguided regulations, and municipalities some of the worst processes, that prevent housing from being affordable to build. Developers can’t be certain to make back their investment when housing is not in crisis, and we need to fix this."
Repeat after me: 80% of the cost of building in SF is land, labor and materials. Meditate on that fact every time you're compelled to repeat something about regulations. When developers in SF complain about regulations, they're dog-whistling to the BMR housing subsidy. That's the bulk of their "regulatory" costs.
Also, stop confusing "California" and San Francisco. They're different. Atherton's building policies have nothing to do with SOMA construction projects.
"The core issue is that we collectively do not have the imagination to enable thousands of small-time landowners to undercut the mega-developers with naturally affordable housing. Zoning makes it illegal, and government process makes it impractical, for ordinary non-developer people to build around here."
Nonsense. Small-time landowners cannot fix housing density in dense, fully-built cities like SF. To create any meaningful amount of housing, you need the capital to buy large amounts of land, labor and materials for high-rise buildings. That's why big developers are doing the construction -- not "regulation", and certainly not "zoning" (while I agree that height limits can/should be higher in many places in SF, it wouldn't suddenly enable small landlords to build skyscrapers. Your "evidence" for this claim has very little to say about zoning; it's merely a collection of snarky graphics).