This is why the free market is so great. 1. Everything too expensive? Good, don't live here, eventually enough people will leave to drive all of the costs back to other market norms. 2. Dumb startups? Good, they will fail, people won't be able to afford to live here and they'll leave. 3. People are spoiled? Uber is cheaper and more convenient in some cases than taking the local transportation. When people stop taking…
You are speaking of a free market that is highly theoretical, because there is no such "free market" in today's economy, no matter where you live. However, free market theory suggests that a situation such as SF is not possible under a free market.
San Francisco has a great deal of political might, which explains a lot of the founding contexts allowing it to survive in the way it does. Add the fact that employees cannot be held to an non-compete agreement, and you have a bunch of engineers stepping over one another to chase the next six month gig as the number of ridiculous startup options balloons to a totally unsustainable size.