The paradox occurs because rent prices are much more flexible than peoples' income. If you have rapid increase in wages (like in SF area), then landlords match the increase by raising rents even faster than the wages rose and the tenants don't get no better services nor quality of living. Just higher prices they have to pay.
That supply and demand determine the price is like describing gravity as "things just fall to the ground when dropped". While true, these tell you nothing about how fast things fall nor where the market prices stabilize.
To see where rent levels come from and why they match up with general income level for a given locality, look at David Ricardo's "law of rent" (introduced in 1809), which tells you exactly that. It basically tells you that rents increase exactly to the level that wages increased. Therefore eating away all technological progress and only giving you higher rents and house prices. This is the progress and poverty paradox. The harder the society works, the larger wealth gap and poverty.
Failing to understand the cause of all this people just slap some random regulation to alleviate the problem - in this case the rent control.
--- To observe the law of rent, simply look at the fact that rents almost exactly match some general level of income of majority of bottom workers so that they barely get by and don't save a cent. Any excess goes to rent.
Just plot wage vs rent of any city on earth and you'll see it.