Basically:
- Traditionally, the proper fix for a shortage of housing is to build more houses. Just lift the existing regulations, and let the free market do what it does best: take care of the demand at an efficient price.
- However, after years and years of very slow building, the amount of pent up demand is such that even if builders were allowed to construct houses, the houses would almost all be built for the very well off. The amount of houses you'd have to build before you started constructing truly affordable housing is immense.
- People that are in rent regulated apartments have absolutely nothing to gain from more construction. However, even poor people that have no stable housing also have almost nothing to gain from new construction because their only realistic chance of having affordable rent is through subsidies or semi-public housing in the medium term.
- The people that have the most to gain are the moderately well off who can afford SF rents, but cannot find housing at the existing market rate.
Basically - politicians and residents let the situation with housing in SF get so absolutely bad that free-market based solutions would take a crazy amount of time to create affordable housing for poor people. At this point, I'm not sure there even are good solutions, especially if you limit yourself to what is politically feasable.
The lesson to draw from this is that you should never let the situation get so bad that people at the bottom stop caring completely about improving policy. In this case, the right policy (build A LOT more houses) would barely help poor people because the supply/demand curve is so out of wack. The very people that should benefit the most from improving the housing situation are instead opposing it - for fairly rational reasons.