Like a background radiation, we have an "absolute background" correlation value...a value we might test against e.g. |+/- .02321| Or we could drop the null
REJECT THE NULL HYPOTHESIS !!! :-)
Everything Is Correlated
21–30 of 56 posts
Re: Everything Is Correlated
#22I think this article is trying to tie two things together, the p-value problem and the fact you can throw in more data.
I disagree.
It's cheating, it's goes against experimental design analysis, and it does not differentiate between given data and data that was carefully collected. We have experimental design class for a reason. It helps us to be honest. Of course there are tons of pit falls many novice statisticians can do.
It also implicitly leads people to think that statistic can magically handle given data and big data by doing the old fashion statistic way. If you do that than of course you'll get a good p-value.
Re: Everything Is Correlated
#23Earlier quoted context omitted.
Ipso factum all "natural" variables are related to bounded random walk which produces clusters (Markovian process), or otherwise have complex chaotic (e.g. fractal) mechanics, which also produces clusters. This follows from physics. Maximum entropy as well as zero entropy is a very rare state to observe.
does this imply that the universe somehow rewards structures that engender 'compressibility' (coarse graining)? it does seem like our brains subjectively enjoy identifying it, to the point of over-optimization in the form of phenomena like pareidolia
The universe doesn’t “reward” it so much as it’s just a consequence of random events. For example, if you flip a coin many times, you’ll see long sequences of heads. From the central limit theorem it follows that sufficiently many random events will form a normal distribution, which exhibits clustering phenomenon. Take a look at a Galton board in action.
Re: Everything Is Correlated
#24>"The fact that these variables are all typically linear or additive further implies that interactions between variables will be typically rare or small or both (implying that most such hits will be false positives, as interactions are far harder to detect than main effects)."
Where does this "fact" come from? And if everything is correlated with everything else all these effects are true positives...
Also, another ridiculous aspect of this is that when data becomes cheap the researchers just make the threshold stricter so it doesn't become too easy. They are (collectively) choosing what is "significant" or not and then acting like "significant" = real and "non-significant" = 0.
Finally, I didn't read through the whole thing. Does he claim to have found an exception to this rule at any point?
Re: Everything Is Correlated
#25I think this article is trying to tie two things together, the p-value problem and the fact you can throw in more data. I disagree. It's cheating, it's goes against experimental design analysis, and it does not differentiate between given data and data that was carefully collected. We have experimental design class for a reason. It helps us to be honest. Of course there are tons of pit falls many novice statisticians…
> It's cheating, it's goes against experimental design analysis, and it does not differentiate between given data and data that was carefully collected. We have experimental design class for a reason. It helps us to be honest. Of course there are tons of pit falls many novice statisticians can do.
Explicit sequential testing runs into exactly the same problem. The problem is, the null hypothesis is not true. So no matter whether you use fixed (large) sample sizes or adaptive procedures which can terminate early while still preserving (the irrelevant) nominal false-positive error rates, you will at some sample size reject the null as your power approaches 100%.
Re: Everything Is Correlated
#26I think this article is trying to tie two things together, the p-value problem and the fact you can throw in more data. I disagree. It's cheating, it's goes against experimental design analysis, and it does not differentiate between given data and data that was carefully collected. We have experimental design class for a reason. It helps us to be honest. Of course there are tons of pit falls many novice statisticians…
> It's cheating, it's goes against experimental design analysis, and it does not differentiate between given data and data that was carefully collected. We have experimental design class for a reason. It helps us to be honest. Of course there are tons of pit falls many novice statisticians can do. Explicit sequential testing runs into exactly the same problem. The problem is, the null hypothesis is not true. So no ma…
This is mostly right, but you are still thinking of these rejections as "false positives" for some reason. They are real deviations from the null hypothesis ("true positives"). The problem is the user didn't test the null model they wanted, it is 100% user error.
Re: Everything Is Correlated
#27Agree that NHST using simple null hypothesis of the form H0: μ = 0 doesn't provide much value. H0 is never true, and the conclusion of "rejecting H0" based on a p-value is therefore not super profound. Also "rejecting H0" conclusion doesn't really tells anything about the alternative hypothesis HA (not even considered when computing p-value, since p-value is under H0). Dichotomies in general are bad, but NHST with po…
That still gives 50-50 odds with sufficient sample size, not much of a test of the research hypothesis (since many alternatives will predict the same direction). It is better than 100% chance of rejection though.
Re: Everything Is Correlated
#28It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. That's why we tell students to consider both p values (which you could think of as a form of quality control on the dataset) and variance explained. Loftus and Loftus make the point nicely: p tells you if you have enough samples and any effect to consider, variance explained tells you if it's worth purs…
> "It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. "
Where does Fisher point this out?
> "That's why we tell students to consider both p values (which you could think of as a form of quality control on the dataset)"
How is this "quality control"? It just tells you whether your sample size was large enough to pass an arbitrary threshold...
Re: Everything Is Correlated
#29>" The fact that these variables are all typically linear or additive further implies that interactions between variables will be typically rare or small or both (implying that most such hits will be false positives, as interactions are far harder to detect than main effects)." Where does this "fact" come from? And if everything is correlated with everything else all these effects are true positives... Also, another…
> Finally, I didn't read through the whole thing. Does he claim to have found an exception to this rule at any point?
Oakes 1975 points out that explicit randomized experiments, which test a useless intervention such as school reform, can be exceptions. (Oakes might not be quite right here, since surely even useless interventions have some non-zero effect, if only by wasting peoples' time & effort, but you might say that the 'crud factor' is vastly smaller in randomized experiments than in correlational data, which is a point worth noting.)
Re: Everything Is Correlated
#30It's well known that the number of Nicholas Cage movies is correlated with a wide variety of natural phenomena.