Moving to a World Beyond "p < 0.05" (2019)
tandfonline.com
Moving to a World Beyond "p < 0.05" (2019)
1–10 of 111 posts
Re: Moving to a World Beyond "p < 0.05" (2019)
#2[deleted]
Re: Moving to a World Beyond "p < 0.05" (2019)
#3As someone who has studied genetics on my own for the last twenty years I am very glad to read this editorial.
For example, take a population of 100 people, and let us say one of them has gene changes in their Fatty Acid Desaturase genes (FADS1 an d FADS2) that change how important Long Chain Omega 3 Fatty Acids (like from fish) are for them. This happens more often in people from indigenous arctic populations.
https://www.sciencedirect.com/science/article/pii/S000291652...
So the researcher tests if omega 3 effects cardiovascular outcome in these hundred people by adding a lot more fish oil to the diet of these 100 people. Since only one of them really needs it, the P value will be insignificant and everyone will say fish oil does nothing. Yet for that one person it was literally everything.
This is talked about only quietly in research, but I think the wider population needs to understand this to know how useless p Interpreting Clinical Trials With Omega-3 Supplements in the Context of Ancestry and FADS Genetic Variation https://www.frontiersin.org/journals/nutrition/articles/10.3...
Re: Moving to a World Beyond "p < 0.05" (2019)
#4"Don’t base your conclusions solely on whether an association or effect was found to be “statistically significant” (i.e., the p-value passed some arbitrary threshold such as p Don’t believe that an association or effect exists just because it was statistically significant.
Don’t believe that an association or effect is absent just because it was not statistically significant.
Don’t believe that your p-value gives the probability that chance alone produced the observed association or effect or the probability that your test hypothesis is true.
Don’t conclude anything about scientific or practical importance based on statistical significance (or lack thereof)."
Hopefully this can help address the replication crisis[0] in (social) science.
[0]: https://en.wikipedia.org/wiki/Replication_crisis
Edit: Formatting (sorry formatting is hopeless).
Re: Moving to a World Beyond "p < 0.05" (2019)
#5As someone who has studied genetics on my own for the last twenty years I am very glad to read this editorial. For example, take a population of 100 people, and let us say one of them has gene changes in their Fatty Acid Desaturase genes (FADS1 an d FADS2) that change how important Long Chain Omega 3 Fatty Acids (like from fish) are for them. This happens more often in people from indigenous arctic populations. https…
From a methods perspective, wouldn't this be more of a statistical power issue (too small of sample size) than a random effect issue? Granted, we do a terrible job discussing statistical power.
Re: Moving to a World Beyond "p < 0.05" (2019)
#6As someone who has studied genetics on my own for the last twenty years I am very glad to read this editorial. For example, take a population of 100 people, and let us say one of them has gene changes in their Fatty Acid Desaturase genes (FADS1 an d FADS2) that change how important Long Chain Omega 3 Fatty Acids (like from fish) are for them. This happens more often in people from indigenous arctic populations. https…
From a methods perspective, wouldn't this be more of a statistical power issue (too small of sample size) than a random effect issue? Granted, we do a terrible job discussing statistical power.
This effect would be even more pronounced in a larger sample size. Consider how small a fraction indigenous arctic populations are of the population as a whole. In other words, larger sample sizes would be even worse off in this particular occasion.
Re: Moving to a World Beyond "p < 0.05" (2019)
#7As someone who has studied genetics on my own for the last twenty years I am very glad to read this editorial. For example, take a population of 100 people, and let us say one of them has gene changes in their Fatty Acid Desaturase genes (FADS1 an d FADS2) that change how important Long Chain Omega 3 Fatty Acids (like from fish) are for them. This happens more often in people from indigenous arctic populations. https…
From a methods perspective, wouldn't this be more of a statistical power issue (too small of sample size) than a random effect issue? Granted, we do a terrible job discussing statistical power.
The best way to talk about this is IMO effect heterogeneity. Underlying that you have the causal DAG to consider, but that‘s (a) a lot of effort and (b) epistemologically difficult!
Re: Moving to a World Beyond "p < 0.05" (2019)
#8"Don’t base your conclusions solely on whether an association or effect was found to be “statistically significant” (i.e., the p-value passed some arbitrary threshold such as p Don’t believe that an association or effect exists just because it was statistically significant. Don’t believe that an association or effect is absent just because it was not statistically significant. Don’t believe that your p-value gives th…
> Hopefully this can help address the replication crisis[0] in (social) science.
I think it isn't just p-hacking.
I've participated in a bunch of psychology studies (questionaires) for university and I've frequently had situations where my answer to some question didn't fit into the possible answer choices at all. So I'd sometimes just choose whatever seems the least wrong answer out of frustration.
It often felt like the study author's own beliefs and biases strongly influence how studies are designed and that might be the bigger issue. It made me feel pretty disillusioned with that field, I frankly find it weird they call it a science. Although that is of course just based on the few studies I've seen.
Re: Moving to a World Beyond "p < 0.05" (2019)
#9As someone who has studied genetics on my own for the last twenty years I am very glad to read this editorial. For example, take a population of 100 people, and let us say one of them has gene changes in their Fatty Acid Desaturase genes (FADS1 an d FADS2) that change how important Long Chain Omega 3 Fatty Acids (like from fish) are for them. This happens more often in people from indigenous arctic populations. https…
The real underlying problem is that in your case, genetic variants are not accounted for. As soon as you include these crucial moderating covariates, it‘s absolutely possible to find true effects even for (rather) small samples (one out of a hundred is really to few for any reasonable design unless it‘s longitudinal)
Re: Moving to a World Beyond "p < 0.05" (2019)
#10Earlier quoted context omitted.
From a methods perspective, wouldn't this be more of a statistical power issue (too small of sample size) than a random effect issue? Granted, we do a terrible job discussing statistical power.
This effect would be even more pronounced in a larger sample size. Consider how small a fraction indigenous arctic populations are of the population as a whole. In other words, larger sample sizes would be even worse off in this particular occasion.
But it is more complicated. I have Sami Heritage but it goes back to my great great grandparents. I did not know this until I started digging deeply into my ancestry, but I carry many of the polymorphisms from these people.
So although I look like a typical European Caucasian, my genetics are very untypical of that population. And this also explains my family history of heart diseases and mood disorders which are also non-typical of European Caucasians.