Live data from Hacker News

When correlation is better than causation

narrator.ai

11–20 of 41 posts

Re: When correlation is better than causation

#11
Isn't the problem here that things are backward?

You don't prove causation, but you can disprove it when you find absence of correlation.

Observed correlation suggests causation which allows you to make a prediction. A prediction can be tested. The prediction will either be true or false based upon whether the correlation continues to hold.

This is one of the problems with A/B tests--they often don't have causation aka "Why?" "This dialog box was rearranged and gave us 15% better conversion." Um. Okay. But "Why?" If you can't answer "Why?" you don't have causation.

"We removed needing to enter a phone number and now have 15% better conversion." "Why?" is obvious in that case.

Re: When correlation is better than causation

#12
post #11

Isn't the problem here that things are backward? You don't prove causation, but you can disprove it when you find absence of correlation. Observed correlation suggests causation which allows you to make a prediction. A prediction can be tested. The prediction will either be true or false based upon whether the correlation continues to hold. This is one of the problems with A/B tests--they often don't have causation a…

A/B tests can tell you causation. If they are correctly randomized and the tested change doesn't induce any other unintended changes, then they can tell you that the change causes whatever are the observed changes in your target metric. The challenge is controlling all other factors.

You're correct that they can't tell you the the root cause of why your change causes a particular difference, but that's a separate issue from correlation and causation.

Re: When correlation is better than causation

#13
post #11

Isn't the problem here that things are backward? You don't prove causation, but you can disprove it when you find absence of correlation. Observed correlation suggests causation which allows you to make a prediction. A prediction can be tested. The prediction will either be true or false based upon whether the correlation continues to hold. This is one of the problems with A/B tests--they often don't have causation a…

I don’t think this post is trying to use correlation to prove causation. It’s in effect saying that when you can’t be sure that there is a causal relationship between two things that you can still make some decisions.

Perfect is the enemy of good as they say.

Re: When correlation is better than causation

#14

Tl;Dr: never, but causality is hard to establish much of the time, so sometimes we must do without. To be honest, I don't find this very convincing. Most of the insights seem pretty obvious. Like if you're working from the point of correlating totals across differently sized legs of an experiment, you're starting from a really bad place. Personally, I'm not quite positive that I buy that causation is that hard to est…

>More often than not, when stakeholders require "causality" to make a decision, it takes way too long so they lose patience and end up making a decision without any data at all.

And therein, I believe, lies the problem.

I think the issue is the pressure for science to produce something constantly so in today's world, correlation is causality. Whether or not you believe in deterministic laws that govern reality, correlation is often the easiest approach when looking at a difficult problem and there in lies the rise of much of probabilistic and statistical models in the face of difficulty. Not all cases, but a lot of cases. We don't want to continue trying the hard work of determining definitive casual relations, if they exist and are content with correlative relations.

As someone who grew up fascinated by science because it was science that sought and provided causal relations, I'm often disappointed about the current world of research. I'm not saying this work is easy by any means, it just seems like we often give up anymore after we pick up the low hanging fruit.

Re: When correlation is better than causation

#15
> The reality is that causality is very difficult to prove. Not only does it require a higher level of statistical rigor, it also requires A LOT of carefully collected data. Meaning you will have to wait a long time before you can make any causal claim.

Malcolm Gladwell's similar message: https://www.pushkin.fm/episode/burden-of-proof/

- A correlation between mining and lung cancer was discovered in 1918, but wasn't acted on until 1975.

- There is a correlation between football and suicide/brain damage, but it is not being acted on.

Re: When correlation is better than causation

#16
post #14

Tl;Dr: never, but causality is hard to establish much of the time, so sometimes we must do without. To be honest, I don't find this very convincing. Most of the insights seem pretty obvious. Like if you're working from the point of correlating totals across differently sized legs of an experiment, you're starting from a really bad place. Personally, I'm not quite positive that I buy that causation is that hard to est…

> More often than not, when stakeholders require "causality" to make a decision, it takes way too long so they lose patience and end up making a decision without any data at all. And therein, I believe, lies the problem. I think the issue is the pressure for science to produce something constantly so in today's world, correlation is causality. Whether or not you believe in deterministic laws that govern reality, corr…

I don't think it's necessarily that scientists are avoiding the hard work to show causality. It's that the most interesting causal experiments are often unethical, or the independent variable cannot be hidden like a placebo (so the participants' bias affect the randomization), or it's simply impossible.

I'll use one example from some data I've been looking at, which is whether the covid-19 pandemic has changed how people sleep. To study this using the formal notion of causality requires asking a random 50% of people to sleep as if covid isn't happening. That's obviously both impractical and implausible.

So you can really only look at correlations. But I can show you the correlations, and I bet you will be convinced that the pandemic HAS changed peoples' sleep. Here's some charts if you can take a look: https://jeffhuang.com/covid_sleep/ but there's probably several factors that convince you that this is causal.

First is the pattern of sleep pre-covid is very stable, and feels trustworthy because it goes up and down during weekends, and holidays are visible. So the data is visibly sensitive to changes in the environment. Second, nearly every country reacts similarly when the N is separated, so even if there's some large group of people somewhere that are outliers (say, some policy by California that everyone needs to go to bed later), it would only affect that one country they are in, not each country separately the same way. Finally, the patterns of sleep post-covid are also stable with similar patterns as pre-covid, but just shifted.

I'm not sure if there's formal ways of representing these concepts, but I feel humans understand these intuitively.

Re: When correlation is better than causation

#17
post #4

https://en.wikipedia.org/wiki/Abductive_reasoning > [Abductive reasoning] starts with an observation or set of observations and then seeks the simplest and most likely conclusion from the observations. This process, unlike deductive reasoning, yields a plausible conclusion but does not positively verify it. Abductive conclusions are thus qualified as having a remnant of uncertainty or doubt, which is expressed in ret…

There are some good papers which take inspiration from abductive reasoning to study automatic knowledge base construction and structured inference here in case anyone is interested: https://www.cs.utexas.edu/~ml/publications/area/65/abduction

Re: When correlation is better than causation

#18
post #15

> The reality is that causality is very difficult to prove. Not only does it require a higher level of statistical rigor, it also requires A LOT of carefully collected data. Meaning you will have to wait a long time before you can make any causal claim. Malcolm Gladwell's similar message: https://www.pushkin.fm/episode/burden-of-proof/ - A correlation between mining and lung cancer was discovered in 1918, but wasn't…

That’s no exactly true. I think the NFL has moved past denial

https://www.espn.com/nfl/story/_/id/22603654/nfl-doctor-says...

https://www.today.com/parents/brett-favre-psa-urges-no-tackl...

Whether the game can ever be made safe is another issue.

Re: When correlation is better than causation

#19
post #11

Isn't the problem here that things are backward? You don't prove causation, but you can disprove it when you find absence of correlation. Observed correlation suggests causation which allows you to make a prediction. A prediction can be tested. The prediction will either be true or false based upon whether the correlation continues to hold. This is one of the problems with A/B tests--they often don't have causation a…

A/B tests can tell you causation. If they are correctly randomized and the tested change doesn't induce any other unintended changes, then they can tell you that the change causes whatever are the observed changes in your target metric. The challenge is controlling all other factors. You're correct that they can't tell you the the root cause of why your change causes a particular difference, but that's a separate iss…

[deleted]

Re: When correlation is better than causation

#20

Tl;Dr: never, but causality is hard to establish much of the time, so sometimes we must do without. To be honest, I don't find this very convincing. Most of the insights seem pretty obvious. Like if you're working from the point of correlating totals across differently sized legs of an experiment, you're starting from a really bad place. Personally, I'm not quite positive that I buy that causation is that hard to est…

[deleted]
Post reply on HN