His basic point is almost a tautology. Yes, it's great to own productive assets. The real trick is predicting which ones will really stay productive over the long term. Just buying a broad index is often not a winning strategy when you factor in the taxes, inflation, and survivor bias. Certainly Buffet hasn't just bought a broad index -- he makes highly targeted investments.
And he glosses over a basic point that I've seen many commenters miss. An asset like gold behaves differently under bubble conditions than an asset like a tech stock, tulip, or house. The production of those things is price elastic, and rising prices cause rising production that ultimately crashes the bubble.
But expanding the supply of gold available for trade by more than a couple percent a year simply isn't feasible. This is precisely why markets have repeatedly chosen it as a form of money.
Of course this means the market for gold is driven purely by demand sentiment, and of course demand could crash. But this is true of every currency -- it's only valuable because it's valuable.