Time to Abolish "Statistical Significance"?

#1

Post by swibbler » Fri, Sep 13, 2019, 1:36 AM UTC

Time to Abolish "Statistical Significance"?
conversableeconomist.blogspot.com

Re: Time to Abolish "Statistical Significance"?

#2

Post by yosefzeev » Fri, Sep 13, 2019, 5:16 PM UTC

I suspect if you take away tenure being based upon publication, you will find that many statistical measures become more honest. You can abolish statistical significance, but it won't stop the abuse of knowledge, which is the real problem here anyway.

Re: Time to Abolish "Statistical Significance"?

#3

Post by microcolonel » Fri, Sep 13, 2019, 5:24 PM UTC

yosefzeev wrote: post #2

I suspect if you take away tenure being based upon publication, you will find that many statistical measures become more honest. You can abolish statistical significance, but it won't stop the abuse of knowledge, which is the real problem here anyway.

Pre-registration is becoming more popular, and I think it will have a massive positive effect w.r.t. abuse of low-grade statistical tests and thresholds.

Re: Time to Abolish "Statistical Significance"?

#4

Post by kalium-xyz » Fri, Sep 13, 2019, 5:31 PM UTC

Why is 0.05 so common to begin with. Its an arbitrary value ain't it?

Re: Time to Abolish "Statistical Significance"?

#5

Post by paulddraper » Fri, Sep 13, 2019, 5:35 PM UTC

That was a lot of words.

There are two alternatives to the current methodology:

* Remove significance requirement for publishing

* Adopt another statistical measure like Bayesian stats

Re: Time to Abolish "Statistical Significance"?

#6

Post by paulddraper » Fri, Sep 13, 2019, 5:37 PM UTC

kalium-xyz wrote: post #4

Why is 0.05 so common to begin with. Its an arbitrary value ain't it?

It's as arbitrary as the age 18.

Not a magic number, just a threshold with consensus.

Re: Time to Abolish "Statistical Significance"?

#7

Post by lisper » Fri, Sep 13, 2019, 5:38 PM UTC

kalium-xyz wrote: post #4

Why is 0.05 so common to begin with. Its an arbitrary value ain't it?

Yep. But if you're going to make binary publishing decisions then you have to draw a line somewhere.

Re: Time to Abolish "Statistical Significance"?

#8

Post by s1artibartfast » Fri, Sep 13, 2019, 5:54 PM UTC

paulddraper wrote: post #5

That was a lot of words. There are two alternatives to the current methodology: * Remove significance requirement for publishing * Adopt another statistical measure like Bayesian stats

There are a few more listed in the article and elsewhere: * pre-registered experiments * listing number of regressing models * P-values with no significance declaration

Re: Time to Abolish "Statistical Significance"?

#9

Post by [deleted] » Fri, Sep 13, 2019, 5:54 PM UTC

[deleted]

Re: Time to Abolish "Statistical Significance"?

#10

Post by kazinator » Fri, Sep 13, 2019, 6:23 PM UTC

lisper wrote: post #7

kalium-xyz wrote: post #4

Why is 0.05 so common to begin with. Its an arbitrary value ain't it?

Yep. But if you're going to make binary publishing decisions then you have to draw a line somewhere.

However "1/20 chance that, if the hypothesis is false, our research confirms it anyway" is not a good choice of somewhere for any important research that can affect people's lives.