21–30 of 142 posts
If you're looking for a replacement you don't understand the problem. The problem isn't that P=.05 is an arbitrary measure of significance. The problem is that only publishing significant results is a bias against the null hypothesis . Let's say you're doing a study of flipping coins. The null hypothesis is that the coin is evenly weighted. If the null hypothesis is true, when you flip a coin once, it will come up he…
Major problem with "significance" is that in some areas of research (say psychology), it's possible to gather lots of data(let 1000 people fill complex questionnaire) and from that data to fish for theory that's significant (in your sample Republicans might have been dumber than Democrats). But given the size of the sample and number of theories you test, you are bound to find something significant even if that isn't…
Not a statistician myself but in a visualization project we did a number of years ago that needed a quick fix for this we used https://en.wikipedia.org/wiki/Bonferroni_correction .
If you're looking for a replacement you don't understand the problem. The problem isn't that P=.05 is an arbitrary measure of significance. The problem is that only publishing significant results is a bias against the null hypothesis . Let's say you're doing a study of flipping coins. The null hypothesis is that the coin is evenly weighted. If the null hypothesis is true, when you flip a coin once, it will come up he…
Good explanation! A typo: "requirement of P<.5" → "requirement of P<.05".
The problem isn't p-values, the problem is a binary distinction between p=0.049 and p=0.051. The problem would go away if everyone understood p-values, or we replaced use of the term "statistically significant" with "3% probability we're just seeing a pattern by accident". Renaming the term to something that sounds just as binary isn't any different.
Scientists have a duty to communicate what is true and what is not though. Passing that judgement duty to the general public is irresponsible and unwise (it's easy to claim anything with flawed / misleading statistics). Ideally someone would have come up with a framework that is better than statistic tests to justify levels of truth, but so far we dont have one
Earlier quoted context omitted.
How about replication?
Replication until statistically significance is found?
nobody has solved the trust issue
Major problem with "significance" is that in some areas of research (say psychology), it's possible to gather lots of data(let 1000 people fill complex questionnaire) and from that data to fish for theory that's significant (in your sample Republicans might have been dumber than Democrats). But given the size of the sample and number of theories you test, you are bound to find something significant even if that isn't…
What I predict would happen: - either lots of studies are allowed to preregister, most of which cannot reject the null hypothesis. You end up with a lot of "boring" null-result papers in those high-profile journals that nobody gets excited about and nobody gets promoted for and no media coverage happens, bad marketing for universities and research centers.
- or there would be a strict filter for the pre-registration, so that researchers cannot chase their gut intuitions, some authority would need to approve the study even before it's done. This hinders research and hinders the dissemination of truly unexpected discoveries.
The human incentives are way deeper than any one solution could touch on. The whole science funding structure requires flashy and sexy results that are just not possible to produce on that scale.
The problem isn't p-values, the problem is a binary distinction between p=0.049 and p=0.051. The problem would go away if everyone understood p-values, or we replaced use of the term "statistically significant" with "3% probability we're just seeing a pattern by accident". Renaming the term to something that sounds just as binary isn't any different.
The problem isn't p-values, the problem is a binary distinction between p=0.049 and p=0.051. The problem would go away if everyone understood p-values, or we replaced use of the term "statistically significant" with "3% probability we're just seeing a pattern by accident". Renaming the term to something that sounds just as binary isn't any different.
Earlier quoted context omitted.
Scientists have a duty to communicate what is true and what is not though. Passing that judgement duty to the general public is irresponsible and unwise (it's easy to claim anything with flawed / misleading statistics). Ideally someone would have come up with a framework that is better than statistic tests to justify levels of truth, but so far we dont have one
When you've got empirically gathered data, pretty much every type of analysis you can do with it comes under the umbrella of "statistics". I think any better truth measuring framework can only be statistical.
If you're looking for a replacement you don't understand the problem. The problem isn't that P=.05 is an arbitrary measure of significance. The problem is that only publishing significant results is a bias against the null hypothesis . Let's say you're doing a study of flipping coins. The null hypothesis is that the coin is evenly weighted. If the null hypothesis is true, when you flip a coin once, it will come up he…