Live data from Hacker News

Correlation is usually not causation. But why not?

gwern.net

21–30 of 74 posts

Re: Correlation is usually not causation. But why not?

#21
post #6

Earlier quoted context omitted.

That's great if you know a priori that you cannot predict the outcome of what you are betting with.

I don't think it would be necessary to know a priori that something is unpredictable; failing to find a pattern after some number of observations should allow an algorithm to determine that the event is not predictable by that algorithm.

The trouble is that you can't reliably distinguish random data (or data determined by factors you're not considering) from patterned data.

Re: Correlation is usually not causation. But why not?

#22
post #10

Bad winter weather can cause auto accidents, and we expect a positive correlation between bad winter storms and winter auto accidents. Okay, but in the northern hemisphere, living in more northern latitudes also correlates with winter auto accidents but does not cause them. For heart disease, we know that the main causes have to do with aging. Well, then, since now the audience for TV news is comparatively old, we ca…

>Well, then, since now the audience for TV news is comparatively old, we can expect that watching TV news has positive correlation with heart disease. Still watching TV news does not cause heart disease.

Well, I'm not so sure. All this sitting to watch TV news, plus all the stress from bad news and fear-mongering...

Re: Correlation is usually not causation. But why not?

#23
post #15
post #8

[deleted]

The author more than understands this. In the first two paragraphs he address the much more interesting question about whether, in realistic causal DAGs, potential correlations grow at a faster rate than actual causal links—the idea being that if they did it would harm the notion that seeing a correlation improves the hope of causation due to pigeonholing. The author doesn't assert that this is true, but it definitel…

[deleted]

Re: Correlation is usually not causation. But why not?

#24
post #21

Earlier quoted context omitted.

I don't think it would be necessary to know a priori that something is unpredictable; failing to find a pattern after some number of observations should allow an algorithm to determine that the event is not predictable by that algorithm.

The trouble is that you can't reliably distinguish random data (or data determined by factors you're not considering) from patterned data.

How do you think random number generators are tested?

Re: Correlation is usually not causation. But why not?

#25
The article is a little dense for me, and presumes knowledge about Probabilistic Graphical Models (PGMs) and directed acyclic graphs / causal Bayesian networks (DAGs) without introducing the background knowledge - so, I expect my reading of it missed a lot of the details.

But the one thing I came out wondering, is if you do a proper randomized double-blinded study with a large population sample, say, taking 1000 people with a particular illness, completely randomizing them, and then giving 500 of the sample a particular treatment, and 500 of the sample a placebo that is indistinguishable from the treatment. If an unbiased third-party, then observes the groups (still without knowledge of which group had which treatment), and if one group shows markedly different results (say, 490/500 of the treatment group recover, and only 10/500 placebo group recover), is it fair to say that in this particular scenario, that correlation of recovery with the treatment implied that the treatment caused the recovery?

Re: Correlation is usually not causation. But why not?

#28
All I know about causation and correlation I learnt hunting bugs in large legacy software systems. In that environment I got the impression that correlation almost never equalled causation, but that's only because the hardest bugs, the ones I remembered, were hard because the obvious correlations did not help identify the root cause. A similar argument might be made for scientific studies: most of the easy causes that can be identified from correlation have already been found, leaving the majority of new studies with correlations that don't easily establish causation.

Re: Correlation is usually not causation. But why not?

#29

The article is a little dense for me, and presumes knowledge about Probabilistic Graphical Models (PGMs) and directed acyclic graphs / causal Bayesian networks (DAGs) without introducing the background knowledge - so, I expect my reading of it missed a lot of the details. But the one thing I came out wondering, is if you do a proper randomized double-blinded study with a large population sample, say, taking 1000 peop…

Yes. In this case, you have two outcomes: Recovering and Not Recovering. You also have two starting states: Drug and Placebo (which both came off of the original starting state, Participant in Study, with a 50% probability).

So your Drug -> Recover has a 98% probability, vs. Drug -> Not Recover of 2%. Likewise, Placebo -> Recover has only a 2% probability, vs. Placebo -> Not Recover of 98%. In this case, you know with very high probability that the fork you stuck in the road caused two very clear, disparate outcomes. Especially because you actually began the "give drug" phase.

However, usually medical trials have results more along the lines of: Placebo group of 35 participants: 10 recovered in < 5 days, 12 recovered in < 10 days, and 13 failed to recover. Drug group of 35 participants: 17 recovered in < 5 days, 13 recovered in < 10 days, and 5 failed to recover. With data like that, clearly SOMETHING causes people to recover, and clearly your drug is not the underlying cause of recovery. It may be supporting whatever process is causing recovery, but it's very unclear whether that's even the case.

Re: Correlation is usually not causation. But why not?

#30
post #7

This question is bound to be a can of worms. There has been a great deal written about the matter, particularly in regard to observational studies. Drawing causal inference is overwhelmingly likely to be wrong when there is a good chance that unknown variables are influencing the correlates observed. In health-related sciences that more often than not is the case. A few years ago there was a study correlating hours o…

> There was a great article published in PLOS several years ago (ATM I don't have the link) showing mathematically that the odds were about 1 in a million that an observational study like the above would turn out to be a "true" causal relationship, and the author concluded most published studies were junk.

I very much doubt the second part of that, that most published studies are junk. The idea that not showing a causal link and only a correlational one makes a study junk is not held by anyone in the field who garners a whole lot of respect or notoriety. I'm not saying the quality is equal whatsoever, simply that correlation studies are not inherently junk - some are fantastic and some are quite the opposite.

There is no question that journals in general are pumping out a significant amount of junk (in studies of all types), but I speculate the root cause of that has more to do with the rise of "publish or perish" and significant increases in grad school enrollment. And even worse the notion that for a grad student, the number of publications they have their name attributed to is more important than the content they publish in terms of employment after they graduate. So there is a situation where people have more pressure to publish than ever before [0], there is more competition for scarce funding so studies are vastly underfunded and studies are rushed so another can begin to add to the resume.

That doesn't mean there is less good science being done either! There probably is more good science being published now than ever before, the problem is the signal:noise ratio has gone down making it harder for good studies to get media attention, and easier for the media to latch on to whatever story they think will get viewers.

[0]: Sidenote, this also makes it increasingly challenging not only to have high quality research, but research that only is making a correlation as opposed to going through and providing evidence to claim you might have a causal link.

Post reply on HN