Two explanations, both of them centered around markets being made of different participants who all have different incentives, but the price being set on the margin of the market:
1.) Prices change over time. Someone who paid $2.4M for a 13-unit apartment block (as our landlord did) collects over $400K/year in rent at $2600/unit/month. That's a 17% return, well above interest, taxes, and property management. That same building might sell for $10M now, which would be a 4% return, barely enough to cover interest, let alone taxes and management. Indeed, new luxury apartment blocks are often renting for $7600/month+ (and they tend to get filled by Googlers on corporate business or new to the area - see point 2).
2.) Different people want different things, and so the pool of buyers and renters are often completely disjoint, with different means. I was shocked to discover that the whole 13-unit complex I'm in sold for just $2.4M (granted, 2012, when prices weren't as insane; see point #1). That's less than some of my friends are paying for single-family homes. And you're right, it's not exactly equivalent: it's multi-family, so there are neighbors, and no real yard (but it does have a common-space lawn). But would I pay triple the housing costs to not have neighbors? No - they don't annoy me that much.
The thing is, by being willing to live in multi-family housing, you change the population of people who are competing with you for housing. Most of the neighbors here are Mexican immigrants, blue-collar workers, or tech-workers who are doing the same thing we're doing (sacrificing some standard of living to save money). We aren't competing with other folks who have Google or Apple stock options, we're competing with folks who are making minimum wage and packing 3-4 families into a unit to make ends meet. In other words, you can save a lot of money by not caring about status and appearance.
(Actually, after Zillowing nearby prices, it makes me wonder if we should just buy a small duplex or multi-family apartment block, live in one unit, and rent out the other ones. Rent often nearly covers the full mortgage, and with a large down payment, it starts getting profitable quickly. There's still the issue of the housing stock in California being pretty terrible, though.)