Also, the larger a team is, the more the average ability of the team will approach the average ability of the population it recruits from.
But the reality is that big teams have problems independently of the capacity of the companies to attract talent. So I don't think that "average ability" is the source of the problem.
As someone else posted, The Mythical Man Month is a good read on this area. And communication overhead is an unsolvable problem as the people involved intro a project grows.