How to Think about Correlation?
statmodeling.stat.columbia.edu
How to Think about Correlation?
1–10 of 60 posts
Re: How to Think about Correlation?
#2Correlation and convolution are adjoints.
Re: How to Think about Correlation?
#3The article's question: "Why are correlations meaningful?" Here's a simplified answer.
Correlations are meaningful because they indicate how strongly variables can predict other variables, in some circumstances. This is valuable because some variables are easier to use than others.
A positive correlation between two variables means they tend to move in the same direction (relative to their mean).
A negative correlation between two variables means they tend to move in opposite directions.
A strong correlation between two variables means that they tend to change in much the same kinds of relative magnitudes, and also that either variables can be used to generally predict the other.
A weak correlation between two variables means that the two variables tend to change in somewhat the same kinds of relative magnitudes, sometimes with some kinds of exceptions, and also that either variable can be used to somewhat predict the other.
Thus a strong positive correlation means that variables can predict each other effectively, and also move in the same direction. This is especially valuable for projects where one variable is easy to obtain, yet the other variable is hard to obtain.
And it turns out that a strong positive correlation can be a good place to hunt for possible causality, such as hunting for something that causing both variables to change together. However, correlation doesn't imply causation.
A strong positive correlation can be good evidence that a hypothesized causality is likely to be true. For example an experiment can test if X causes Y, by varying X, and discovering that X and Y tend to change together.
Edit: the replies below are great-- thank you-- well worth reading. I'm editing my post to make it clearer that it's a simplified answer, and to add suggestions by the replies.
Re: How to Think about Correlation?
#4The article's question: "Why are correlations meaningful?" Here's a simplified answer. Correlations are meaningful because they indicate how strongly variables can predict other variables, in some circumstances. This is valuable because some variables are easier to use than others. A positive correlation between two variables means they tend to move in the same direction (relative to their mean). A negative correlati…
I don't quite like this answer, because it simplifies a lot of things about correlations which are just not really accurate.
> Corrleations are meaningful because they indicate how strongly variables can predict other variables.
This is not wrong per se, but may be wrongly interpreted.
1) I think it is important to stress that the reverse is not true. If a correlation is 0, it does not mean the variables are independent. It means they are linearly independent. There are a lot of things out there which are very much linked but not with a linear relationship.
2) This is only true if the correlation was done properly. For instance, correlating non-stationary variables will spuriously yield a high correlation, giving a false impression of link. The classic example is trying to correlate stock prices between themselves. An other example of spurious correlation is when not enough points are given to get an accurate result. Looking at tstat, r2, etc. is as important as looking at the actual correlation coefficient.
3) You use the word "predict" here, which I don't quite like. Correlation can predict outcomes only in the case where you have proven that past information will continue to occur. This is not always the case.
> A positive correlation between two variables means they tend to move in the same direction.
This is misleading also. The variables tend to move in the same direction _relatively to their mean_! I can plot you very different looking curves that are highly correlated just because their mean vary differently. Especially because of potential outliers.
> And it turns out that a strong positive correlation can be a good place to hunt for possible causality, such as hunting for something that causing both variables to change together.
I tend to dislike that thought process. It is very tempting to infer causality from correlation. Especially when automating correlation at scale is affordable, so generating spurious correlation is easy. To me correlation is more a way to reinforce a hypothesis of causality that is already suspected. It won't give you the direction, but may hint you that you are on the right track. I mean, correlation should not be a clue to discover causality, as there are too many false positives. Correlation should be an intermediary step in studying a potential causality.
Re: How to Think about Correlation?
#5Correlation and convolution are adjoints.
Can you expand a little further? I don't see the adjunction (between what categories?)
Re: How to Think about Correlation?
#6The article's question: "Why are correlations meaningful?" Here's a simplified answer. Correlations are meaningful because they indicate how strongly variables can predict other variables, in some circumstances. This is valuable because some variables are easier to use than others. A positive correlation between two variables means they tend to move in the same direction (relative to their mean). A negative correlati…
I don't quite like this answer, because it simplifies a lot of things about correlations which are just not really accurate. > Corrleations are meaningful because they indicate how strongly variables can predict other variables. This is not wrong per se, but may be wrongly interpreted. 1) I think it is important to stress that the reverse is not true. If a correlation is 0, it does not mean the variables are independ…
Thank you for this. I learnt probability from Sheldon Ross. I now realise I don't grok correlations at all! I had the exact same mistakes in my mental model of correlation as OP. Where do I go (what do I read) to get a good account of correlation?
Re: How to Think about Correlation?
#7The article's question: "Why are correlations meaningful?" Here's a simplified answer. Correlations are meaningful because they indicate how strongly variables can predict other variables, in some circumstances. This is valuable because some variables are easier to use than others. A positive correlation between two variables means they tend to move in the same direction (relative to their mean). A negative correlati…
> And it turns out that a strong positive correlation can be a good place to hunt for possible causality
I don't disagree, but I have yet to see a mathematically clean definition of "causality".
Re: How to Think about Correlation?
#8Imagine my surprise when I realized that correlation and synchrony are (almost) the same thing.
Re: How to Think about Correlation?
#9The article's question: "Why are correlations meaningful?" Here's a simplified answer. Correlations are meaningful because they indicate how strongly variables can predict other variables, in some circumstances. This is valuable because some variables are easier to use than others. A positive correlation between two variables means they tend to move in the same direction (relative to their mean). A negative correlati…
> And it turns out that a strong positive correlation can be a good place to hunt for possible causality I don't disagree, but I have yet to see a mathematically clean definition of "causality".
Do you take issue with Pearl?
Re: How to Think about Correlation?
#10Just to add another reason to think of correlation as linear models: spearman correlation is analogous to fitting a linear model on the ranks of the inputs rather than using the values themselves.