The reason for San Francisco's high rent/home price is very simple, despite how others like to complicate it more: 1: Its desirable for most people to live here. Possibly the most desirable place in the USA, depending on your tastes. As a result, people are willing to pay more to live here. There are jobs, culture, food, art and much more all in this great city. 2: Rent Control. I wont get too much into it, but my ta…
People who are upwardly mobile follow the jobs. In general, they choose jobs, then cities. Except for the very rich who don't have to work, desirability (excluding commute) is a minor factor in prices. I can back that up, but I think most people buy this already.
However, real estate demand is extremely inelastic. If 5 percent of the houses in Manhattan were destroyed, the price increase wouldn't be 5%. It would probably be 2-5x, possibly 10x. That's why, even though rent control only affects a small percentage of the market, its price influence is huge.
New development is supposed to correct for that, but often the result of it is creation of more expensive housing, which solves quality problems but doesn't have the desired price effect until many years later. (Real estate rarely goes down in dollar-denominated amounts, for psychological reasons; what does happen in a soft market is that prices stay flat and lose to inflation. In effect, it's a loss-limit on price declines that is very tight when inflation is low.)