Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything Is Correlated

gwern.net

Post reply on HN
Print view View original

41–50 of 56 posts

Re: Everything Is Correlated

#41

Post by AstralStorm »

throwawaymath wrote: post #23
ChainOfFools wrote: post #18

Earlier quoted context omitted.

does this imply that the universe somehow rewards structures that engender 'compressibility' (coarse graining)? it does seem like our brains subjectively enjoy identifying it, to the point of over-optimization in the form of phenomena like pareidolia

The universe doesn’t “reward” it so much as it’s just a consequence of random events. For example, if you flip a coin many times, you’ll see long sequences of heads. From the central limit theorem it follows that sufficiently many random events will form a normal distribution, which exhibits clustering phenomenon. Take a look at a Galton board in action.

That's ignoring anything related to actual life we observe and Gaussian distributed data does not have to exhibit clustering either. (But it allows that.)

About the only thing that is naturally uniform so far within bounds is large scale homogeneity and isotropy of universe. Which is an unsolved mystery potentially involving dark matter.

Re: Everything Is Correlated

#42

Post by zeusk »

tcgv wrote: post #11
ajuc wrote: post #7

When the correlation is close to 0 it's often because of a feedback loop. For example - in economy with central bank trying to hit inflation target - interest rates and inflation will have near 0 correlation (interest rates change but inflation remains constant). That's because central bank adjusts interest rates to counter other variables so that inflation remains near the target. Other example (my favorite, it was…

That's very interesting. In the car driving example we can define three variables: 1) Throttle 2) Speed 3) Elevation derivative If "3" is constant (ex: flat terrain) then "1" and "2" will have strong correlation. However if "2" is constant (ex: cruise control) as in your example, "1" and "3" will have strong correlation. In the economic example, however, this kind of analisys should be much more complex and take plen…

The key point being identifying those variables and ensuring they remain constant (i.e. in that example - tire pressure, elevation, fuel load etc.)

Re: Everything Is Correlated

#43

Post by gwern »

nonbel wrote: post #28
limbicsystem wrote: post #5

It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. That's why we tell students to consider both p values (which you could think of as a form of quality control on the dataset) and variance explained. Loftus and Loftus make the point nicely: p tells you if you have enough samples and any effect to consider, variance explained tells you if it's worth purs…

> "It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. " Where does Fisher point this out? > "That's why we tell students to consider both p values (which you could think of as a form of quality control on the dataset)" How is this "quality control"? It just tells you whether your sample size was large enough to pass an arbitrary threshold...

> Where does Fisher point this out?

Probably in the Fisher excerpt.

Re: Everything Is Correlated

#44

Post by gwern »

nonbel wrote: post #35
gwern wrote: post #29

Earlier quoted context omitted.

> Finally, I didn't read through the whole thing. Does he claim to have found an exception to this rule at any point? Oakes 1975 points out that explicit randomized experiments, which test a useless intervention such as school reform, can be exceptions. (Oakes might not be quite right here, since surely even useless interventions have some non-zero effect, if only by wasting peoples' time & effort, but you might say…

Thanks, How about this "fact": The fact that these variables are all typically linear or additive ?

That is simply a corollary of the fact that Pearson's r and regressions are usually linear/additive, and things like Meehl's demonstration wouldn't work if they weren't. You'd just calculate all the pairwise correlations and get nothing if they were solely totally nonlinear/interactions. (In which case you'd have a hard time proving they were related at all.)

Re: Everything Is Correlated

#45

Post by MadWombat »

ajuc wrote: post #7

When the correlation is close to 0 it's often because of a feedback loop. For example - in economy with central bank trying to hit inflation target - interest rates and inflation will have near 0 correlation (interest rates change but inflation remains constant). That's because central bank adjusts interest rates to counter other variables so that inflation remains near the target. Other example (my favorite, it was…

I like the gas pedal example. I read a similar one somewhere where we measure temperature inside the house and energy usage by the heater. The energy usage is correlated with outside temperature, but inside temperature stays constant, so we conclude that inside temperature is unrelated to heater and turn the heater off.

Re: Everything Is Correlated

#46

Post by nonbel »

gwern wrote: post #43
nonbel wrote: post #28

Earlier quoted context omitted.

> "It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. " Where does Fisher point this out? > "That's why we tell students to consider both p values (which you could think of as a form of quality control on the dataset)" How is this "quality control"? It just tells you whether your sample size was large enough to pass an arbitrary threshold...

> Where does Fisher point this out? Probably in the Fisher excerpt.

I looked but did not see it.

Re: Everything Is Correlated

#47

Post by nonbel »

gwern wrote: post #44
nonbel wrote: post #35

Earlier quoted context omitted.

Thanks, How about this "fact": The fact that these variables are all typically linear or additive ?

That is simply a corollary of the fact that Pearson's r and regressions are usually linear/additive, and things like Meehl's demonstration wouldn't work if they weren't. You'd just calculate all the pairwise correlations and get nothing if they were solely totally nonlinear/interactions. (In which case you'd have a hard time proving they were related at all.)

> You'd just calculate all the pairwise correlations and get nothing if they were solely totally nonlinear/interactions.

I don't believe this. Most nonlinear correlations also show up as non-zero (linear) correlation coefficients. There are really only a couple pathological cases I can think of where it would not happen.

Re: Everything Is Correlated

#48

Post by pierrebai »

Is this trying to be too clever? If the correlation is weaker than the random noise of the data, then it is equivalent to not being correlated.

Otherwise, we'd get conclusions like the color of your car influencing your risk of lung cancer or some such nonsense. With enough data, you could see a weak correlation of red car to cancer, but it would still be insignificant. That's what the null-hypothesis is for: to put a treshold under which we can just ignore whatever weak correlation seems to be there.

Re: Everything Is Correlated

#49

Post by ajuc »

seiferteric wrote: post #20
ajuc wrote: post #7

When the correlation is close to 0 it's often because of a feedback loop. For example - in economy with central bank trying to hit inflation target - interest rates and inflation will have near 0 correlation (interest rates change but inflation remains constant). That's because central bank adjusts interest rates to counter other variables so that inflation remains near the target. Other example (my favorite, it was…

I really like that example, but I am wondering if it would really be true? Real drivers would not maintain a perfect speed, but would instead work to maintain the average. If you looked closely at the speed, it would drift away from the average, then the peddle would move to return it to the average. So it would look a bit like an integral (The I in PID control) of the difference from the mean speed right?

Yup, that's how you know I only had this on university, never used it in real life :) I think in real life you might see the feedback loop in motion, or not, depending on the resolution and sampling.

Re: Everything Is Correlated

#50

Post by dalore »

AstralStorm wrote: post #41
throwawaymath wrote: post #23

Earlier quoted context omitted.

The universe doesn’t “reward” it so much as it’s just a consequence of random events. For example, if you flip a coin many times, you’ll see long sequences of heads. From the central limit theorem it follows that sufficiently many random events will form a normal distribution, which exhibits clustering phenomenon. Take a look at a Galton board in action.

That's ignoring anything related to actual life we observe and Gaussian distributed data does not have to exhibit clustering either. (But it allows that.) About the only thing that is naturally uniform so far within bounds is large scale homogeneity and isotropy of universe. Which is an unsolved mystery potentially involving dark matter.

I would argue if it didn't do clustering then there was some sort of pattern/bias at play that caused it.
Post reply on HN