Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seven basic rules for causal inference

pedermisager.org

Post reply on HN
Print view View original

31–40 of 71 posts

Re: Seven basic rules for causal inference

#31

Post by currymj »

cubefox wrote: post #28
dash2 wrote: post #22

Earlier quoted context omitted.

The standard mathematical definition of correlation means linear correlation. If you are talking about non-independence, it would be better to use that language. This early mistake made me think the author is not really an expert.

What is an appropriate measure of (in)dependence though, if not Pearson correlation? Such that you feed a scatter plot into the formula for this measure, and if the measure returns 0 dependence, the variables are independent.

it's a tough problem.

there are various schemes for estimating mutual information from samples. if you do that and mutual information is very close to zero, then I guess you can claim the two rvs are independent. But these estimators are pretty noisy and also often computationally frustrating (the ones I'm familiar with require doing a bunch of nearest-neighbor search between all the points).

I agree with the OP that it's better to say "non-independence" and avoid confusion, at the same time, I disagree that linear correlation is actually the standard definition. In many fields, especially those where nobody ever expects linear relationships, it is not and everybody uses "correlated" to mean "not independent".

Re: Seven basic rules for causal inference

#32

Post by cubefox »

bdjsiqoocwk wrote: post #10
apwheele wrote: post #9

Earlier quoted context omitted.

This was my thought as well. I don't like showing the scatterplots in these examples, as "correlation" I think is more associated with the correlation coefficient than the more generic independence that the author means in this scenario. E.g. a U shape in the scatterplot may have a zero correlation coefficient but is not conditionally independent.

> E.g. a U shape in the scatterplot may have a zero correlation coefficient but is not conditionally independent. Ok this is correct, but has nothing to do with causality. Whether or not two variables are correlated and whether or not they are independent, and when one does or doesn't imply the other, is a conversation that can be had without resorting to the concept of causality at all. And in fact that's how the su…

> Ok this is correct, but has nothing to do with causality.

It does. Dependence and independence have a lot to do with causation, as the article explains.

> Whether or not two variables are correlated and whether or not they are independent, and when one does or doesn't imply the other, is a conversation that can be had without resorting to the concept of causality at all.

Yes, but this is irrelevant. It's like saying "whether or not someone is married is a conversation that can be had without resorting to the concept of a bachelor at all".

You can talk about (in)dependence without talking about causation, but you can't talk in detail about causation without talking about (in)dependence.

Re: Seven basic rules for causal inference

#33

Post by dash2 »

rlpb wrote: post #29
dash2 wrote: post #22

Earlier quoted context omitted.

The standard mathematical definition of correlation means linear correlation. If you are talking about non-independence, it would be better to use that language. This early mistake made me think the author is not really an expert.

That seems a bit harsh. People can independently become experts without being familiar with the terminology used by existing experts. Further, if intended for a non-expert audience, it may even be deliberate to loosen definitions of terms used by experts, and being precise by leaving a note about that instead, which apparently is exactly what this author did.

It's much better to use vocabulary consistently with what everyone else does in the field. Then you don't need to add footnotes correcting yourself. And if you are not familiar with what everyone else means by correlation, you're very unlikely to be an expert. This is not like that Indian mathematician who reinvented huge chunks of mathematics.

Re: Seven basic rules for causal inference

#34

Post by geye1234 »

jerf wrote: post #21
lordnacho wrote: post #2

This is brilliant. The whole causal inference thing is something I only came across after university, either I missed it or it is a hole in the curriculum, because it seems incredibly fundamental to our understanding of the world. The thing that made be read into it was a quite interesting sentence from lesswrong, saying that actually the common idea that correlation does not imply causation is wrong. Now it's not wr…

"correlation does not imply causation is wrong" That's a specific instance of a more general problem in the "logical fallacies", which is that most of them are written to be true in an absolutist, Aristotelian frame. It is true that if two things are correlated you can not therefore infer a rigidly 100% chance that there is a causative relationship there. And that's how Aristotelian logic works; everything is either…

I don't disagree with the substance of your comment, but want to clarify something.

Lesswrong promulgated a seriously misleading view of Aristole as some fussy logician who never observed reality and was unaware of probability, chance, the unknown, and so on. It is entirely false. Aristotle repeats, again and again and again, that we can only seek the degree of certainty that is appropriate for a given subject matter. In the Ethics, perhaps his most-read work, he says this, or something like it, at least five times.

I mention this because your association of the words "absolutist" and "Aristotelian" suggests your comment may have been influenced by this.

ISTM that there are two entirely different discussions taking place here, not opposed to each other. "Aristotelian" logic tends to be more concerned with ontology -- measles causes spots, therefore if he has measles, then he will have spots. Whereas the question of probability is entirely epistemological -- we know he has spots, which may indicate he has measles, but given everything else we know about his history and situation this seems unlikely; let's investigate further. Both describe reality, and both are useful.

So the fallacies are entirely fallacious: I don't think your point gainsays this. But I agree that, to us, B may suggest A, and it is then that the question of probability comes into play.

Aquinas, who was obviously greatly influenced by Aristotle, makes a similar point somewhere IIRC (I think in SCG when he's explaining why the ontological argument for God's existence fails), so it's not as if this is a new discovery.

Re: Seven basic rules for causal inference

#35

Post by rlpb »

dash2 wrote: post #33
rlpb wrote: post #29

Earlier quoted context omitted.

That seems a bit harsh. People can independently become experts without being familiar with the terminology used by existing experts. Further, if intended for a non-expert audience, it may even be deliberate to loosen definitions of terms used by experts, and being precise by leaving a note about that instead, which apparently is exactly what this author did.

It's much better to use vocabulary consistently with what everyone else does in the field. Then you don't need to add footnotes correcting yourself. And if you are not familiar with what everyone else means by correlation, you're very unlikely to be an expert. This is not like that Indian mathematician who reinvented huge chunks of mathematics.

> It's much better to use vocabulary consistently with what everyone else does in the field.

Fine, but...

> And if you are not familiar with what everyone else means by correlation, you're very unlikely to be an expert.

Perhaps, but this is not relevant. If there's a problem with this work, then that problem can be criticized directly. There is no need, and it is not useful, to infer "expertise" by indirect means.

Re: Seven basic rules for causal inference

#36

Post by jdhwosnhw »

abeppu wrote: post #30

At the bottom, the author mentions that by "correlation" they don't mean "linear correlation", but all their diagrams show the presence or absence of a clear linear correlation, and code examples use linear functions of random variables. They offhandedly say that "correlation" means "association" or "mutual information", so why not just do the whole post in terms of mutual information? I think the main issue with tha…

This isnt a correction to your post, but a clarification for other readers: correlation implies dependence, but dependence does not imply correlation. Conversely, two variables share non-zero mutual information if and only if they are dependent.

Re: Seven basic rules for causal inference

#37

Post by islewis »

jdhwosnhw wrote: post #36
abeppu wrote: post #30

At the bottom, the author mentions that by "correlation" they don't mean "linear correlation", but all their diagrams show the presence or absence of a clear linear correlation, and code examples use linear functions of random variables. They offhandedly say that "correlation" means "association" or "mutual information", so why not just do the whole post in terms of mutual information? I think the main issue with tha…

This isnt a correction to your post, but a clarification for other readers: correlation implies dependence, but dependence does not imply correlation. Conversely, two variables share non-zero mutual information if and only if they are dependent.

Could you give some examples of dependence without correlation?

Re: Seven basic rules for causal inference

#38

Post by cubefox »

boxfire wrote: post #24
jerf wrote: post #21

Earlier quoted context omitted.

"correlation does not imply causation is wrong" That's a specific instance of a more general problem in the "logical fallacies", which is that most of them are written to be true in an absolutist, Aristotelian frame. It is true that if two things are correlated you can not therefore infer a rigidly 100% chance that there is a causative relationship there. And that's how Aristotelian logic works; everything is either…

I liked the way Pearl phrased it originally. A calculus of anti-correlations implies causation. That makes the nature of the analysis clear and doesn't set of the classic minds alarm bells.

Unfortunately this calculus is exceedingly complicated and I haven't even seen a definition of "a causes b" in terms of this calculus. One problem is that Pearl and others make use of the notion of "d-separation". This allows for elegant proofs but is hard to understand. I once found a paper which replaced d-separation with equivalent but more intuitive assumptions about common causes, but I since forgot the source.

By the way, there is also an alternative to causal graphs, namely "finite factored sets" by Scott Garrabrant. Probably more alternatives exist. Though I don't know more about (dis)advantages.

Re: Seven basic rules for causal inference

#39

Post by xtacy »

islewis wrote: post #37
jdhwosnhw wrote: post #36

Earlier quoted context omitted.

This isnt a correction to your post, but a clarification for other readers: correlation implies dependence, but dependence does not imply correlation. Conversely, two variables share non-zero mutual information if and only if they are dependent.

Could you give some examples of dependence without correlation?

You can check the example described here: https://stats.stackexchange.com/questions/644280/stable-viol...

Judea Pearl’s book also goes into the above in some detail, as to why faithfulness might be a reasonable assumption.

Re: Seven basic rules for causal inference

#40

Post by abeppu »

islewis wrote: post #37
jdhwosnhw wrote: post #36

Earlier quoted context omitted.

This isnt a correction to your post, but a clarification for other readers: correlation implies dependence, but dependence does not imply correlation. Conversely, two variables share non-zero mutual information if and only if they are dependent.

Could you give some examples of dependence without correlation?

A clear graphical set of illustrations is the bottom row in this famous set: https://en.wikipedia.org/wiki/Correlation#/media/File:Correl...

They have clear dependence; if you imagine fixing ("conditioning") x at a particular value and looking at the distribution of y at that value, it's different from the overall distribution of y (and vice versa). But the familiar linear correlation coefficient wouldn't indicate anything about this relationship.

Post reply on HN