Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything Is Correlated

gwern.net

Post reply on HN
Print view View original

11–20 of 56 posts

Re: Everything Is Correlated

#11

Post by tcgv »

ajuc wrote: post #7

When the correlation is close to 0 it's often because of a feedback loop. For example - in economy with central bank trying to hit inflation target - interest rates and inflation will have near 0 correlation (interest rates change but inflation remains constant). That's because central bank adjusts interest rates to counter other variables so that inflation remains near the target. Other example (my favorite, it was…

That's very interesting. In the car driving example we can define three variables: 1) Throttle 2) Speed 3) Elevation derivative

If "3" is constant (ex: flat terrain) then "1" and "2" will have strong correlation. However if "2" is constant (ex: cruise control) as in your example, "1" and "3" will have strong correlation.

In the economic example, however, this kind of analisys should be much more complex and take plenty of variables into account.

Re: Everything Is Correlated

#12

Post by AstralStorm »

dalore wrote: post #10
mojomark wrote: post #8

Earlier quoted context omitted.

Good discussion. On the flip side, in my data mining class the professor keeps saying ~"you may be able to find clusters in a data set, but often no true correlation exists." However, that's an absolute statement I just don't swallow. In my mind what I see is that if an unexplained correlation or non-correlation appears, it may be random (or true) or it could be the result of an unmeasured (hidden) variable. In your…

Perhaps they are trying to explain clustering illusion? The phenomenon that even random data will produce clusters. You can take that further and state random data WILL produce clusters. If you don't have clusters then your data is not random and some pattern is at play. This really tricks up our mind as our mind tries to find patterns everywhere. If you try and plot random dots you will usually put dots without clus…

Ipso factum all "natural" variables are related to bounded random walk which produces clusters (Markovian process), or otherwise have complex chaotic (e.g. fractal) mechanics, which also produces clusters. This follows from physics.

Maximum entropy as well as zero entropy is a very rare state to observe.

Re: Everything Is Correlated

#14

Post by bluGill »

mojomark wrote: post #8
ajuc wrote: post #7

When the correlation is close to 0 it's often because of a feedback loop. For example - in economy with central bank trying to hit inflation target - interest rates and inflation will have near 0 correlation (interest rates change but inflation remains constant). That's because central bank adjusts interest rates to counter other variables so that inflation remains near the target. Other example (my favorite, it was…

Good discussion. On the flip side, in my data mining class the professor keeps saying ~"you may be able to find clusters in a data set, but often no true correlation exists." However, that's an absolute statement I just don't swallow. In my mind what I see is that if an unexplained correlation or non-correlation appears, it may be random (or true) or it could be the result of an unmeasured (hidden) variable. In your…

The ability for adults to drink milk and fluency in speaking English is well correlated. This is because those of northern European ancestry are more likely to be able to drink milk, and it happens that most of northern European ancestry either immigrated to an English speaking country (US) 150 years ago, or are in a country where English instruction is good.

Re: Everything Is Correlated

#15

Post by rwj »

mjfl wrote: post #6
ivansavz wrote: post #4

Agree that NHST using simple null hypothesis of the form H0: μ = 0 doesn't provide much value. H0 is never true, and the conclusion of "rejecting H0" based on a p-value is therefore not super profound. Also "rejecting H0" conclusion doesn't really tells anything about the alternative hypothesis HA (not even considered when computing p-value, since p-value is under H0). Dichotomies in general are bad, but NHST with po…

Couldn't you make an argument that the point H0 has use when you are testing whether two populations are identical? i.e. it's probably true that \mu is very close to 0 if it is the difference in heights of men from Nebraska vs men from Iowa.

You've kind of hit the point with the second half of your comment. Two populations are virtually never identical, so you don't need any statistics to answer the question. A more reasonable question is whether or not you have the statistical power (i.e. measurement precision) to see the difference, and whether the difference is big enough to matter.

Re: Everything Is Correlated

#16

Post by snthpy »

ajuc wrote: post #7

When the correlation is close to 0 it's often because of a feedback loop. For example - in economy with central bank trying to hit inflation target - interest rates and inflation will have near 0 correlation (interest rates change but inflation remains constant). That's because central bank adjusts interest rates to counter other variables so that inflation remains near the target. Other example (my favorite, it was…

> Other example (my favorite, it was mindblowing when my teacher showed it to us on econometrics as a warning :) ) - gas pedal and speed of a car driving on a hilly road. Driver wants to drive near the speed limit, so he adjusts the gas pedal to keep the speed constant. Simplistic conclusion would be - speed is constant despite the gas pedal position changing therefore they are unrelated :)

I think that's Milton Friedman's Thermostat in case you want to search for it.

Re: Everything Is Correlated

#17

Post by Mistletoe »

This reminds me of the current omnigenic hypothesis about genes. That unexpectedly almost every gene seems to affect the expression of traits.

https://www.quantamagazine.org/omnigenic-model-suggests-that...

"Drawing on GWAS analyses of three diseases, they concluded that in the cell types that are relevant to a disease, it appears that not 15, not 100, but essentially all genes contribute to the condition. The authors suggested that for some traits, “multiple” loci could mean more than 100,000."

Re: Everything Is Correlated

#18

Post by ChainOfFools »

AstralStorm wrote: post #12
dalore wrote: post #10

Earlier quoted context omitted.

Perhaps they are trying to explain clustering illusion? The phenomenon that even random data will produce clusters. You can take that further and state random data WILL produce clusters. If you don't have clusters then your data is not random and some pattern is at play. This really tricks up our mind as our mind tries to find patterns everywhere. If you try and plot random dots you will usually put dots without clus…

Ipso factum all "natural" variables are related to bounded random walk which produces clusters (Markovian process), or otherwise have complex chaotic (e.g. fractal) mechanics, which also produces clusters. This follows from physics. Maximum entropy as well as zero entropy is a very rare state to observe.

does this imply that the universe somehow rewards structures that engender 'compressibility' (coarse graining)? it does seem like our brains subjectively enjoy identifying it, to the point of over-optimization in the form of phenomena like pareidolia

Re: Everything Is Correlated

#19

Post by gwern »

ajuc wrote: post #7

When the correlation is close to 0 it's often because of a feedback loop. For example - in economy with central bank trying to hit inflation target - interest rates and inflation will have near 0 correlation (interest rates change but inflation remains constant). That's because central bank adjusts interest rates to counter other variables so that inflation remains near the target. Other example (my favorite, it was…

That's a good point. Another one I forgot to make: given the established empirical reality of 'everything is correlated', if you find a variable which does in fact seem to be independent of most or everything else, that alone makes that variable suspicious - it suggests that it may be a pseudo-variable, composed largely or entirely of measurement error/randomness, or perhaps afflicted by a severe selection bias or other problem (such as range restriction or Berkson's paradox eliminating the real correlation).

Somewhat similarly, because 'everything is heritable', if you run into a human trait which is not heritable at all and is precisely estimated at h^2~0, that cast considerable doubt on whether you have a real trait at all. (I've seen this happen to a few latent variables extracted by factor analysis: they have near-zero heritability in a twin study and on further investigation, turn out to have been just sampling error or bad factor analysis in the first place, and don't replicate or predict anything or satisfy any of the criteria you might use to decide if a trait is 'real'.)

Re: Everything Is Correlated

#20

Post by seiferteric »

ajuc wrote: post #7

When the correlation is close to 0 it's often because of a feedback loop. For example - in economy with central bank trying to hit inflation target - interest rates and inflation will have near 0 correlation (interest rates change but inflation remains constant). That's because central bank adjusts interest rates to counter other variables so that inflation remains near the target. Other example (my favorite, it was…

I really like that example, but I am wondering if it would really be true? Real drivers would not maintain a perfect speed, but would instead work to maintain the average. If you looked closely at the speed, it would drift away from the average, then the peddle would move to return it to the average. So it would look a bit like an integral (The I in PID control) of the difference from the mean speed right?
Post reply on HN