The Limits of Expertise

> This seems ...honestly pretty reasonable and well thought out to me.

As someone who has changed a few diapers in my day, it's one of the most idiotic things I've ever read.

A professor writes a children's picture book, about a subject within their field. They are unusually careful about correctness, for they fear the embarrassment of "getting it wrong" in front of their peers. But the book contains an error outside their field, and addressing that, isn't a priority.

So one thing I look for, are the incentives around getting things wrong. It varies at a finer grain than one might expect. Two individuals saying the same tiny thing, may have vastly different associated reputational risks. And an individual's risk, can vary greatly over adjacent subfields and types of claim.

> domains

Imagine a lunch-time party. You overhear an elderly gentleman ask: "So, you are a doctor?" "Yes, of medieval french literature." "Good, I have this question about my digestion...". Did you laugh? Ha! So many people just don't understand how expertise is distributed.

Then across the room, you see a five-year old approaching a group of first-tier astronomy graduate students. She asks them "I have a new set of finger paints! I'm painting space! What color is the Sun? Which paint should I use?". Did you laugh? Ha! How large a group would she need ask, to get even 50-50 odds that they don't all give her the same wrong answer? Silly five-year old - she didn't even ask if any of them had a research focus in spectroscopy! So many people just don't understand how expertise is distributed within the science research community. [From my [1]]

Understanding becomes ramshackle surprisingly rapidly, as you move away from someone's active research focus.

A couple of years back, the PBS News Hour did a "free energy from water in your basement" story. In their later on-air retraction, they seemed defensive. After all, they'd checked the story with, as I fuzzily recall, some "Italian scientists".

I suggest there is widespread misunderstanding about what expertise looks like.

[1] Scientific expertise is not broadly distributed - an underappreciated obstacle to creating better content http://www.clarifyscience.info/part/MHjx6

What I think people are skeptical of today is not necessarily expertise, but credentialism and the idea that expertise should supercede other considerations.

For example, there are a ton of unwarranted assumptions being made when when assumes that someone with an MD knows more about a topic than someone without one, or that someone board certified in an area is more qualified than someone not certified in that area, or that people certified in the area are equally skilled. We substitute weak indicators of skill or aptitude for the actual skill or aptitude, as the indicators are perfect when they are far from it.

Credentials like degrees or background should be treated for what they are: imperfect indicators of expertise. But now we have this situation where the experts are given control and monopolies over domains. In the US healthcare system, for example, MDs are given a strong monopoly over care, to the exclusion of other professions, but is this really warranted? Skepticism about academic economists having a strong role in politics are somewhat similar, in that sure, we should pay attention to them, but should they have quite as strong a role as they do?

I worry also about the tone of these discussions. The problem isn't so much with expertise, for example, it's how we use it or overuse it, or grant it monopoly, or imbue it with meaning it doesn't have.

I'm even skeptical of the linked pieces. The injection of mention of genetics in the one article is odd, for example, and to me the arguments reek of bigotry. Overall, their arguments seem to be "well, expertise is unwarranted, except for the fields I'm in, and we all know this expertise derives from my superior genetic endowment in combination with feedback. You'll see."

The problem with the open versus closed system argument is that even in very chaotic, unpredictable domains, there's some level of experience that is worthwhile. In fact, research suggests that one of the problems is that people tend to overestimate their ability to predict; part of the expertise in such domains should be to recognize this. This is legitimate expertise, recognizing the scope of unknown unknowns.

Similarly, it's also important to note that often in such fields, it's not just that things are difficult to predict, it's that we know they are difficult to predict precisely because so many reasonable, sophisticated things have been tried and failed. If anything, this points to the importance of expertise, because it suggests that very reasonable sophisticated ideas can still be wrong. The author likes to point to physics as an example of a closed system, which is ironic, because certain areas of physics are well outside our current level of understanding. Should we then, say reject physics as a domain of legitimate expertise because we don't currently have a good understanding of the nature of time?