Live data from Hacker News

Moving to a World Beyond "p < 0.05" (2019)

tandfonline.com

31–40 of 111 posts

Re: Moving to a World Beyond "p < 0.05" (2019)

#31
The 0.05 threshold is indeed arbitrary, but the scientific method is sound.

A good researcher describes their study, shows their data and lays their own conclusions. There is just no need (nor possibility) of a predefined recipe to resume the study result into a "yes" or a "no".

Research is about increasing knowledge; marketing is about labelling.

Re: Moving to a World Beyond "p < 0.05" (2019)

#32
post #8

Earlier quoted context omitted.

> Hopefully this can help address the replication crisis[0] in (social) science. I think it isn't just p-hacking. I've participated in a bunch of psychology studies (questionaires) for university and I've frequently had situations where my answer to some question didn't fit into the possible answer choices at all. So I'd sometimes just choose whatever seems the least wrong answer out of frustration. It often felt lik…

> the study author's own beliefs and biases strongly influence how studies are designed While studies should try to be as "objective" as possible, it isn't clear how this can be avoided. How can the design of a study not depend on the author's beliefs? After all, the study is usually designed to test some hypothesis (that the author has based on their prior knowledge) or measure some effect (that the author thinks ex…

If you get an answer outside of what you expected, reevaluate your approach, fix your study and redo it all, probably with a new set of participants.

If you can't do science, don't call it science.

Re: Moving to a World Beyond "p < 0.05" (2019)

#33
post #31

The 0.05 threshold is indeed arbitrary, but the scientific method is sound. A good researcher describes their study, shows their data and lays their own conclusions. There is just no need (nor possibility) of a predefined recipe to resume the study result into a "yes" or a "no". Research is about increasing knowledge; marketing is about labelling.

...but making conclusions is how you get funding!

Re: Moving to a World Beyond "p < 0.05" (2019)

#34

Earlier quoted context omitted.

> the study author's own beliefs and biases strongly influence how studies are designed While studies should try to be as "objective" as possible, it isn't clear how this can be avoided. How can the design of a study not depend on the author's beliefs? After all, the study is usually designed to test some hypothesis (that the author has based on their prior knowledge) or measure some effect (that the author thinks ex…

If you get an answer outside of what you expected, reevaluate your approach, fix your study and redo it all, probably with a new set of participants. If you can't do science, don't call it science.

Which is a great idea if we ignore all other issues in academia, e.g. pressure to publish etc. Taking such a hard-line stance I fear will just yield much less science being done.

Re: Moving to a World Beyond "p < 0.05" (2019)

#35
post #31

The 0.05 threshold is indeed arbitrary, but the scientific method is sound. A good researcher describes their study, shows their data and lays their own conclusions. There is just no need (nor possibility) of a predefined recipe to resume the study result into a "yes" or a "no". Research is about increasing knowledge; marketing is about labelling.

> The 0.05 threshold is indeed arbitrary, but the scientific method is sound.

Agreed. A single published paper is not science, a tree data structure of published papers that all build off of each other is science.

Re: Moving to a World Beyond "p < 0.05" (2019)

#36

As someone who has studied genetics on my own for the last twenty years I am very glad to read this editorial. For example, take a population of 100 people, and let us say one of them has gene changes in their Fatty Acid Desaturase genes (FADS1 an d FADS2) that change how important Long Chain Omega 3 Fatty Acids (like from fish) are for them. This happens more often in people from indigenous arctic populations. https…

As a layman who doesn't work with medical studies it always struck me that one of the bits of data that isn't (normally) collected along with everything else is genetic samples of all participants. It should be stored alongside everything else so that if the day comes when genetic testing becomes cheap enough it can be used to provide vastly greater insight into the study's results.

Even something as simple as a few strands of hair sealed in a plastic bag in a filing cabinet somewhere would be better than nothing at all.

Re: Moving to a World Beyond "p < 0.05" (2019)

#37

Earlier quoted context omitted.

> the study author's own beliefs and biases strongly influence how studies are designed While studies should try to be as "objective" as possible, it isn't clear how this can be avoided. How can the design of a study not depend on the author's beliefs? After all, the study is usually designed to test some hypothesis (that the author has based on their prior knowledge) or measure some effect (that the author thinks ex…

If you get an answer outside of what you expected, reevaluate your approach, fix your study and redo it all, probably with a new set of participants. If you can't do science, don't call it science.

And where will the money come from for this second study? What about a third? Fourth?

We live in a money-dependent world. We cannot go without it.

Re: Moving to a World Beyond "p < 0.05" (2019)

#38
post #34

Earlier quoted context omitted.

If you get an answer outside of what you expected, reevaluate your approach, fix your study and redo it all, probably with a new set of participants. If you can't do science, don't call it science.

Which is a great idea if we ignore all other issues in academia, e.g. pressure to publish etc. Taking such a hard-line stance I fear will just yield much less science being done.

> much less science being done

This isn't obviously a bad thing, in the context of a belief that most results are misleading or wrong.

Re: Moving to a World Beyond "p < 0.05" (2019)

#39

"Don’t base your conclusions solely on whether an association or effect was found to be “statistically significant” (i.e., the p-value passed some arbitrary threshold such as p Don’t believe that an association or effect exists just because it was statistically significant. Don’t believe that an association or effect is absent just because it was not statistically significant. Don’t believe that your p-value gives th…

[deleted]
Post reply on HN