The Peculiar Blindness of Experts

You can’t accurately forecast what the weather will be on June 28th 80 years from now, for the same reason you can’t predict if it will be raining at your house at 3:29 pm two days from now, or where some molecule of water will end up in the Mississippi River an hour from now. But you can assuredly make less specific forecasts based on current trends. You know it’s likely to rain at some point in the morning two days from now, that the water molecule will flow roughly downstream, and that average air temperatures will increase if we keep putting CO2 in the atmosphere.

The Dervish House from Ian McDonald has a plot thread based around that idea.

It's usually easy to spot experts less susceptible to this problem (and more open to change) since they'll almost never consider themselves "experts" and avoid the label. They know how susceptible to change their theories could be or potential for being misguided/wrong. Their deep degree of understanding shows them just how little they truly know.

Furthermore, in our society, you must project confidence to be successful. Many experts are forced to appear confident when they're really not (does not make them less of an expert). If you're an expert and don't come off as exact, non-experts (and some experts even) may falsely assume you're not an expert because they expect you can produce an answer to some question they pose with certainty.

I believe this structure is partly to blame for the problem. Some play this confidence charade and are able to separate out the act of appearing certain, but still understanding the uncertainties they don't share and keeping them held tight.

The others get caught in this feedback loop where appearing confident and being right translates emotionally to being more confident. Their confidence is rewarded so they assume the the perspectives they have confidence in are sound the more it occurs over time. When a case comes a long that breaks that flow, they're still confident, but have positive emotional ties to those beliefs and become a victim of their own successes. Something else must be wrong because I've been rewarded so much for being right about this.

I believe those doing translational/applied sciences and engineering are more susceptible to problem than those on the theoretical side due to the way things are incentivized as a society. On the theoretical side there's a greater acceptance of being wrong (that's becoming less true as research is privatized more). On the applied side, people expect a high degree of confidence and certainty because that's what they pay for and there's little-to-no acceptance for error even though we're all human and make errors daily.

So I think the problem is really due to how society views experts and exerts its expectations upon them.

Consider an expert in medicine, back when the "four humours" theory of medicine was what experts used. A non-expert would think, "bleeding is bad, I don't want to bleed, I need my blood". Experts believed in bleeding their patients/victims to restore a balance of the humours. George Washington was probably killed by his doctor in this way.

Being an expert in a field only helps, when the basic theoretical models of that field have reached a certain minimum level of maturity.