Spent several years in San Francisco. Hated it. Moved back to the midwest (Ann Arbor, then back home to Chicago).
What don't I like about SFBay? With the caveat that it's been almost 10 years since I lived there, let me make you a list:
* It's expensive. What that means is, you have fewer options on where to live, and the options aren't as nice.
* Apart from tacos, the casual food options are much worse. Concrete example: where do you go for late dinner after a show, besides a diner in the Castro and Mel's?
* There are very few decent music venues. Among my many complaints about SF, this is one that appears to have gotten much worse since I left; read JWZ's blog on SF's "war on fun".
* It is impossible to catch a cab.
* It is very difficult (and dangerous) to park. My car was towed in San Francisco and lost by the city for over a month.
* San Francisco is deceptively small. Deceptive, because SFBay is a major metro area that happens to sprawl from a small urban core. What this means is that many of your friends who ostensibly live in the same "place" as you are actually an hour's drive each way to get to.
* Similarly, many (technically, most) of the jobs in the "area" are actually a 1+ hour commute each way from your house. At the time, I didn't know anybody who took the train from San Francisco to South Bay; maybe that's changed, but my perception is everyone just drives and puts up with the horrible commute. 280 sure is pretty sometimes, though.
* It's filthy. I've been back recently and know this to still be true: San Francisco has a maintenance problem, and it's different and worse than Manhattan. Manhattan feels like you're always picking your way through someone's gigantic cluttered basement. This alters your expectations of the environment and makes bags of garbage on the street easier to take. San Francisco is flat and open and that makes the piles of human feces somehow harder to take.
* Public transportation in San Francisco, is a joke compared to NYC or Chicago or even Seattle. It achieves Houstonian levels of walkability despite a miniscule footprint.
* The Haight and the stretch of Golden Gate Park right off the Haight is disgusting. I'm in favor of legalization, but if we're going to do it with red light districts, we should at least cordon them off and make that clear.
* Too much of San Francisco is a tourist trap. Every big city has tourist areas, but it's more painful in San Francisco because the city is so small. In NYC and Chicago there are actual Italian restaurant neighborhoods and they aren't full of frat boys.
* San Francisco doesn't have neighborhoods like other cities do. People tend to counter this by saying "but the inner sunset is totally different from the outer sunset". When's the last time they had a block party? How many of their neighbors have they entertained in their houses in the last year?
* There are no seasons (ok, there's "grey" season and "yellow" season). It's about to be Autumn here in Chicago. Autumn in the midwest is amazing and I missed it, a lot.
* Everyone's in tech. This makes the social scene boring and incestuous. Also, unless you build products that serve the echo chamber, it makes it harder to sell to companies, because the industries in San Francisco aren't diverse.
I'm trying hard with this list not to ding San Francisco for things that other cities excel at. It's not reasonable to hate a city for not meeting the highest bars set by other cities. For instance, fine dining in San Francisco is, if you properly exclude Yountville from your definition of "San Francisco", simply not as good as NYC or even Chicago. San Francisco doesn't have world-class museums. It doesn't have a noteworthy theater scene, or (to my knowledge) excellence in any of the performing arts. But then, Chicago doesn't have redwood forests or beaches you can set bonfires on with no notice.