Statisticians want to abandon science’s standard measure of ‘significance’

This is an excellent point. In my work, the interpretation error I see the most among people around be is to confuse significance with the magnitude of the predicted effect. I.e. if you use OLS to create a model, it is possible to have a statistically significant beta, with essentially no magnitude.

If you look at research on carcinogens, it is frequent to see something like "it is statistically significant that eating chemical X increases your risk of cancer" but when you read on, the magnitude of the effect is "increases risk by .0023%, for a cancer that already has a .6% incidence in the population".

My point is that people who are not statisticians (or people who use statistics), tend to be concerned with the magnitude of an effect, and frequently misconstrue the statistical significance for that magnitude.

The problem isn't that the wording is confusing, it's that p(x|H0) isn't a very useful thing to compute.

Everybody wants to compute p(H|x), the probility of a scientific hypothesis given the data. People want to do this so badly that they can't help interpreting the p-value that way.

You can actually compute p(H|x) if you use Bayesian stats.

Here is a related article on NOVA. https://www.pbs.org/wgbh/nova/article/rethinking-sciences-ma...

I disbelieve.

The problem is that people want an answer to the question, "Here is a pile of data, what should I believe?" But mathematically, the proper role of data is to modify existing beliefs, and not to dictate beliefs.

Statisticians can spend forever explaining this. But instead have gone with the cop-out of asking a question that is confusingly similar to the one that people want to ask. It is popular exactly because it is so easily misunderstood. You can give any number of lectures on what it actually means - I guarantee that it will be misunderstood.

Worse yet, p-values are sensitive to particulars of experimental design that logically should never matter to inferences. My aunt and uncle provide a classic example. They wanted a son and a daughter. They had 6 sons then a daughter. Are they biased towards one gender? The p-value for this question is .03125 (we'd see this much evidence against with all boys, all girls, last girl, last boy, so 4/2^7). But if I instead said that they planned to have 7 children, and had 6 sons then a daughter, the p-value changes to 0.125 because the one child out could have been anywhere in succession. But in Bayes' formula, their plans if something else had happened cannot ever affect how we adjust our inferences.

(This example is not made up. When Lorna found that they had a daughter, she told Bill to get a vasectomy from the delivery table. By another crazy coincidence, their 7 children were also born on all the days of the week.)

Well, sort-of.

Its not just about p-values or the likelyhood of making a particular type of error by agreeing with a thesis.

There is a fundamental misunderstanding that most people, and even many scientists make with regards to the philosophical interpretation of 'truth' using the scientific method.

By definition, scientific 'truth' is always fallible, and the vast majority of people have a very hard time dealing with this concept. A better explanation or additional data, or better data is always possible and needs to be possible for science to not be dogma. This is fundamentally different than how people interpret truth and understand what the word means. All scientific truths are replaceable.