Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving to a World Beyond "p < 0.05" (2019)

tandfonline.com

Post reply on HN
Print view View original

81–90 of 111 posts

Re: Moving to a World Beyond "p < 0.05" (2019)

#81

Post by UweSchmidt »

koe123 wrote: post #34
UweSchmidt wrote: post #32

Earlier quoted context omitted.

If you get an answer outside of what you expected, reevaluate your approach, fix your study and redo it all, probably with a new set of participants. If you can't do science, don't call it science.

Which is a great idea if we ignore all other issues in academia, e.g. pressure to publish etc. Taking such a hard-line stance I fear will just yield much less science being done.

Let's do a less science then, but rigorous and throrough. Or find more funding.

But surely let's have a "hard-line stance" on not drowning in BS?

Re: Moving to a World Beyond "p < 0.05" (2019)

#82

Post by wrp »

Skimming through the other articles in that special issue, I thought the most practical advice was in Abandon Statistical Significance[1], which discusses the practicalities of not treating p-values as a threshold.

[1] https://www.tandfonline.com/doi/full/10.1080/00031305.2018.1...

Re: Moving to a World Beyond "p < 0.05" (2019)

#83

Post by johnobrien1010 »

minkzilla wrote: post #79
johnobrien1010 wrote: post #53

Earlier quoted context omitted.

In THEORY yes, but in practice, there are not a ton of journals I think that will actually publish well done research that does not come to some interesting conclusion and find some p<.05. So....

That is what this article is about, changing the expectations of researchers and journals to be willing to publish and read research without p values.

I see. Thanks for clarifying.

Re: Moving to a World Beyond "p < 0.05" (2019)

#84

Post by BoingBoomTschak »

iterance wrote: post #42
mistermann wrote: post #30

Earlier quoted context omitted.

> ethnicity is not biologically realized either What does this mean? Is it contrary to what OP is saying above?

Not exactly. What I mean to say is this: We know there are certain phenotypes that predominantly appear in certain populations, in broad strokes. But while we do have lists of correlates, we don't have good definitions for what an "ethnicity" is biologically, and there is very good reason to believe no satisfactory definition exists. To use OP's example, we know that the gene mentioned is frequently found in the Inui…

Y-DNA and mtDNA haplogroup are good definitions though. They just don't map exactly to vernacular concepts.

Re: Moving to a World Beyond "p < 0.05" (2019)

#85

Post by bjourne »

ChadNauseam wrote: post #67
johnobrien1010 wrote: post #46

Earlier quoted context omitted.

Yeah but without a hardline how would you decide what to publish?

Publishing only significant results is a terrible idea in the first place. Publishing should be based on how interesting the design of the experiment was, not how interesting the result was.

P-value doesn't measure interestingness. If p>0.05 there was no result at all.

Re: Moving to a World Beyond "p < 0.05" (2019)

#86

Post by ttpphd »

Wow so many comments that just don't get the criticism and want to continue with their magical beliefs about null hypothesis testing.

This is a damning criticism of the most common techniques used by my scientists. The answer isn't to shrug and keep doing the same thing as before.

Re: Moving to a World Beyond "p < 0.05" (2019)

#87

Post by thaumasiotes »

bjourne wrote: post #85
ChadNauseam wrote: post #67

Earlier quoted context omitted.

Publishing only significant results is a terrible idea in the first place. Publishing should be based on how interesting the design of the experiment was, not how interesting the result was.

P-value doesn't measure interestingness. If p>0.05 there was no result at all.

Both of those statements are false. Everything has a result. And the p-value is very literally a quantified measure of how interesting a result was. That's the only thing it purports to measure.

"Woman gives birth to fish" is interesting because it has a p-value of zero: under the null hypothesis ("no supernatural effects"), a woman can never give birth to a fish.

Re: Moving to a World Beyond "p < 0.05" (2019)

#88

Post by joe_the_user »

curiousllama wrote: post #66

The core problem with this is: - DON'T is very clear and specific. Don't say "Stat-Sig", don't conclude causal effect, don't conclude anything based on p>0.05. - DO is very vague and unclear. Do be thoughtful, do accept uncertainty, do consider all relevant information. Obviously, thoughtful consideration of all available information is ideal. But until I get another heuristic for "should I dig into this more?" - I'm…

Yeah, it seems like those bullet point have the problem that they don't really contain actionable information.

Here's the way I'd put things - correlation by itself does causation at all. You need correlation plus a plausible model of the world to have a chance.

Now science, at its best, involves building up these plausible models, so a scientist creates an extra little piece of the puzzle and has to be careful also the piece is a plausible fit.

The problem you hit is that the ruthless sink-or-swim atmosphere, previous bad science and fields that have little merit make it easy to be in the "just correlation" category. And whether you're doing a p test or something else doesn't matter.

A way to put is that a scientist has to care about the truth in order to put together all the pieces of models and data in their field.

So the problem is ultimately institutional.

Re: Moving to a World Beyond "p < 0.05" (2019)

#89

Post by burtness »

BoingBoomTschak wrote: post #84
iterance wrote: post #42

Earlier quoted context omitted.

Not exactly. What I mean to say is this: We know there are certain phenotypes that predominantly appear in certain populations, in broad strokes. But while we do have lists of correlates, we don't have good definitions for what an "ethnicity" is biologically, and there is very good reason to believe no satisfactory definition exists. To use OP's example, we know that the gene mentioned is frequently found in the Inui…

Y-DNA and mtDNA haplogroup are good definitions though. They just don't map exactly to vernacular concepts.

Isn't the problem that the vernacular concepts are what counts and they change depending on time and place?

Re: Moving to a World Beyond "p < 0.05" (2019)

#90

Post by truculent »

curiousllama wrote: post #66

The core problem with this is: - DON'T is very clear and specific. Don't say "Stat-Sig", don't conclude causal effect, don't conclude anything based on p>0.05. - DO is very vague and unclear. Do be thoughtful, do accept uncertainty, do consider all relevant information. Obviously, thoughtful consideration of all available information is ideal. But until I get another heuristic for "should I dig into this more?" - I'm…

> But until I get another heuristic for "should I dig into this more?"

Why do you need a heuristic? In what areas are you doing research where you don't have any other intuition or domain knowledge to draw on?

And if you don't have that background, contextual knowledge, are you the right person to be doing the work? Are you asking the right questions?

Post reply on HN