When correlation is better than causation • Hacker News Board

Re: When correlation is better than causation

#21

Post by whyenot » Sat, Aug 28, 2021, 4:01 AM UTC

melling wrote: post #18

Leftium wrote: post #15

> The reality is that causality is very difficult to prove. Not only does it require a higher level of statistical rigor, it also requires A LOT of carefully collected data. Meaning you will have to wait a long time before you can make any causal claim. Malcolm Gladwell's similar message: https://www.pushkin.fm/episode/burden-of-proof/ - A correlation between mining and lung cancer was discovered in 1918, but wasn't…

That’s no exactly true. I think the NFL has moved past denial https://www.espn.com/nfl/story/_/id/22603654/nfl-doctor-says... https://www.today.com/parents/brett-favre-psa-urges-no-tackl... Whether the game can ever be made safe is another issue.

The NFL might have moved past denial, but the NCAA and high school sports don't seem to have traveled very far along that path.

It's profoundly sad that institutions of higher learning are promoting activities that they know can cause brain damage and long term disability, just so they can make money and entertain their alumni.

Re: When correlation is better than causation

#22

Post by Kranar » Sat, Aug 28, 2021, 4:17 AM UTC

bsder wrote: post #11

Isn't the problem here that things are backward? You don't prove causation, but you can disprove it when you find absence of correlation. Observed correlation suggests causation which allows you to make a prediction. A prediction can be tested. The prediction will either be true or false based upon whether the correlation continues to hold. This is one of the problems with A/B tests--they often don't have causation a…

You can neither use correlation to prove causation nor can you use causation to prove correlation. X can cause Y but be uncorrelated to Y, and X can be correlated to Y without being caused by Y.

Re: When correlation is better than causation

#23

Post by brittanymdavis1 » Sat, Aug 28, 2021, 4:42 AM UTC

BalinKing wrote: post #10

I sort of feel like this: correlation is not causality, but they are correlated nonetheless.

Yes right - and we can actually put that information to some use in some practical way. At least that's the goal.

Re: When correlation is better than causation

#24

Post by brittanymdavis1 » Sat, Aug 28, 2021, 4:46 AM UTC

lazyjeff wrote: post #16

Frost1x wrote: post #14

Earlier quoted context omitted.

> More often than not, when stakeholders require "causality" to make a decision, it takes way too long so they lose patience and end up making a decision without any data at all. And therein, I believe, lies the problem. I think the issue is the pressure for science to produce something constantly so in today's world, correlation is causality. Whether or not you believe in deterministic laws that govern reality, corr…

I don't think it's necessarily that scientists are avoiding the hard work to show causality. It's that the most interesting causal experiments are often unethical, or the independent variable cannot be hidden like a placebo (so the participants' bias affect the randomization), or it's simply impossible. I'll use one example from some data I've been looking at, which is whether the covid-19 pandemic has changed how pe…

Right - The point about human intuition is an important one. It's a key part of the approach that the blog talks about. When we start from intuition, we significantly improve our hypotheses about what could be correlated. And because no decision making is ever done in a vacuum, it's suggesting we lean into this and use it to our advantage instead of knocking on correlations all the time.

Re: When correlation is better than causation

#25

Post by brittanymdavis1 » Sat, Aug 28, 2021, 4:49 AM UTC

Leftium wrote: post #15

> The reality is that causality is very difficult to prove. Not only does it require a higher level of statistical rigor, it also requires A LOT of carefully collected data. Meaning you will have to wait a long time before you can make any causal claim. Malcolm Gladwell's similar message: https://www.pushkin.fm/episode/burden-of-proof/ - A correlation between mining and lung cancer was discovered in 1918, but wasn't…

At the end of the day, a correlation analysis will never replace a causal study. So really un-intuitive relationships would likely have never been uncovered with this approach. But I believe most of the questions we're typically asking - especially in a business context - are super intuitive and, even with a big causal study, they are never that "surprising" - don't you agree?

Re: When correlation is better than causation

#26

Post by ahmedelsama » Sat, Aug 28, 2021, 4:51 AM UTC

wahern wrote: post #4

https://en.wikipedia.org/wiki/Abductive_reasoning > [Abductive reasoning] starts with an observation or set of observations and then seeks the simplest and most likely conclusion from the observations. This process, unlike deductive reasoning, yields a plausible conclusion but does not positively verify it. Abductive conclusions are thus qualified as having a remnant of uncertainty or doubt, which is expressed in ret…

I like this a lot. I studied Bayesian mathematic and in my opinion it’s the beat approach to solving these problems. Start with a prior and continue to update your state with measurements. This avoids a lot of the common pitfalls when doing batch ML and getting junk results

Re: When correlation is better than causation

#27

Post by ahmedelsama » Sat, Aug 28, 2021, 4:54 AM UTC

asdfasgasdgasdg wrote: post #2

Tl;Dr: never, but causality is hard to establish much of the time, so sometimes we must do without. To be honest, I don't find this very convincing. Most of the insights seem pretty obvious. Like if you're working from the point of correlating totals across differently sized legs of an experiment, you're starting from a really bad place. Personally, I'm not quite positive that I buy that causation is that hard to est…

I took the blog to mean something a little different. You should pursue causality but not everything is worth the work so start with a faster easier correlation analysis, then it it seems worth it you can test it to see if it behaves as expected.

Working in data, especially in a startup, we often need to make so many decisions and trying to change the culture is good but when it’s a fire then this approach would get us the furthest

Re: When correlation is better than causation

#28

Post by brittanymdavis1 » Sat, Aug 28, 2021, 4:56 AM UTC

Frost1x wrote: post #14

asdfasgasdgasdg wrote: post #2

Tl;Dr: never, but causality is hard to establish much of the time, so sometimes we must do without. To be honest, I don't find this very convincing. Most of the insights seem pretty obvious. Like if you're working from the point of correlating totals across differently sized legs of an experiment, you're starting from a really bad place. Personally, I'm not quite positive that I buy that causation is that hard to est…

> More often than not, when stakeholders require "causality" to make a decision, it takes way too long so they lose patience and end up making a decision without any data at all. And therein, I believe, lies the problem. I think the issue is the pressure for science to produce something constantly so in today's world, correlation is causality. Whether or not you believe in deterministic laws that govern reality, corr…

Unfortunately this pressure - especially in research communities - has led to a lot of p-hacking.

https://www.pnas.org/content/117/24/13386

It's similar in a business context - the "pressure" of finding a causal result (especially in situations using AB testing) lead to poor analysis practices in order to find something significant.

Re: When correlation is better than causation

#29

Post by brittanymdavis1 » Sat, Aug 28, 2021, 5:00 AM UTC

asdfasgasdgasdg wrote: post #2

Tl;Dr: never, but causality is hard to establish much of the time, so sometimes we must do without. To be honest, I don't find this very convincing. Most of the insights seem pretty obvious. Like if you're working from the point of correlating totals across differently sized legs of an experiment, you're starting from a really bad place. Personally, I'm not quite positive that I buy that causation is that hard to est…

> Personally, I'm not quite positive that I buy that causation is that hard to establish in many cases.

Do you mean it's not hard from an analytical point of view / from a practical data gathering perspective - or both?

I've found that needing causation leads to big delays in backlogs, especially when it's required for every insight, but I'm curious if you've seen it to be different.

Re: When correlation is better than causation

#30

Post by ahmedelsama » Sat, Aug 28, 2021, 5:01 AM UTC

asdfasgasdgasdg wrote: post #6

dbt00 wrote: post #5

Earlier quoted context omitted.

starting with correlation and asserting causation is bad, starting with causation and using correlation as weak evidence is good.

Precisely (or to be really specific, starting with a suspicion of causation and using correlation as one piece of weak evidence).

I think the ideal is start with something that you think is causal, then do the discussed correlation analysis then it it looks fruitful, start testing by changing the distribution of the feature and if it is causal then you should see the kpi move in the same direction.

Does that make sense? Smaller steps to make sure you only invest if it is worth it