Earlier quoted context omitted.
It's sometimes interesting to pose a little thought experiment to somebody about what they think they could get done if they had 10 world class developers and practically unlimited resources - certainly the ability to hire or acquired almost anybody or anything within some degree of reason. What could they get done in 10 years? Now multiply that by ~2,000. That's, roughly, the sort of resources Google has had. Kind o…
That thought experiment was why I joined Google after 4 years of working in startups. The communication cost overhead the grandparent mentioned is why I left Google after 5 years. While there, I observed a lot of projects where I was literally working with a room full of world-class people - folks who had given TED talks, folks who had started major open-source projects, folks who had written the "Bible" of their par…
The space of solutions to a design problem is really high dimensional and the quality of the design has many local maxima. So from that point of view it's not too surprising that averaging the features of several designs puts you at a point in the space which is not a local maximum. For a more apples to apples comparison with the averaging of predictions, it might be useful to consider predicting the value of a random variable drawn from a multimodal distribution. In this case it's highly likely that different people will focus their predictions on different modes and averaging several predictions will be worse than taking a single one in isolation.