Because it's a hard job that not a lot of people can actually do, but most companies still operate under the, now outmoded, model of industrial production. Under this model, every employee's role can be modeled as a fungible cog in an assembly line. This lets management, and organizations as a whole, abstract over their production process and not deal with the particularities of any given worker.
I've got a kind of fuzzy idea that this is a similar mental bias/logical fallacy to the black swan events that Taleb talks about. People assume normal distributions, but the underlying is a Pareto or Power Law distribution. They do this because it's easy and has lots of nice and practical properties. In the case of markets/finance, it makes the math really easy to work with on a day to day basis in the 99.9% case. But in the black swan event, the .1% case, it totally falls apart and your model cannot account for it.
Much like the financier assuming normal instead of Power Law, the production of software assumes the industrial model of production. This works for some high % of cases to get some amount of productivity out of most writers of software. But at the high end of software professionals, or in a situation in which something actually new is being built to solve an unsolved problem, this breaks down.
Anyway, this is not a wholly thought out thesis and mostly rambling, but it is poking at my mind and I figured I'd write it down somewhere.