Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Causal inference as a blind spot of data scientists

dzidas.com

Post reply on HN
Print view View original

11–20 of 106 posts

Re: Causal inference as a blind spot of data scientists

#11

Post by tomrod »

mjburgess wrote: post #10
tomrod wrote: post #4

I've self-learned for a long time in the causal inference space and model evaluation is a concern for me. My biggest concern is falsification of hypotheses. In ML, you have a clear mechanism to check estimation/prediction through holdout approaches. In classical metrics, you have model metrics that can be used to define reasonable rejection regions for hypothesis tests. But causal inference doesn't seem to have this,…

To be clear, the mechanism for checking ML doesn't really check ML. There's really little value in a confidence interval conditional on the same experimental conditions that produced the dataset on which the model is trained. I'd often say it's actively harmful, since it's mostly misleading. Insofar as causal inference has no such 'check', its because there never was any. Casual inference is about dispelling that ill…

> Insofar as causal inference has no such 'check', its because there never was any. Casual inference is about dispelling that illusion.

Aye, and that's the issue I'm trying to understand. How to know if model 1 or model 2 is more "real" or, for my lack of a better term, more useful and reflective of reality?

We can focus on a particular philosophical point, like parsimony / Occam's razor, but as far as I can tell that isn't always sufficient.

There should be some way to determine a model's likelihood of structure beyond "trust me, it works!" If there is, I'm trying to understand it!

Re: Causal inference as a blind spot of data scientists

#12

Post by nyssos »

tomrod wrote: post #11
mjburgess wrote: post #10

Earlier quoted context omitted.

To be clear, the mechanism for checking ML doesn't really check ML. There's really little value in a confidence interval conditional on the same experimental conditions that produced the dataset on which the model is trained. I'd often say it's actively harmful, since it's mostly misleading. Insofar as causal inference has no such 'check', its because there never was any. Casual inference is about dispelling that ill…

> Insofar as causal inference has no such 'check', its because there never was any. Casual inference is about dispelling that illusion. Aye, and that's the issue I'm trying to understand. How to know if model 1 or model 2 is more "real" or, for my lack of a better term, more useful and reflective of reality? We can focus on a particular philosophical point, like parsimony / Occam's razor, but as far as I can tell tha…

> How to know if model 1 or model 2 is more "real" or, for lack of a better term, more useful and reflective of reality?

You don't. Given observational data alone, it's typically only possible to determine which d-separation equivalence class you're in. Identifying the exact causal structure requires intervening experimentally.

> There should be some way to determine a model's likelihood of structure

Why? If the information isn't there, it isn't there. No technique can change that.

Re: Causal inference as a blind spot of data scientists

#13

Post by kjkjadksj »

riedel wrote: post #7
bertil wrote: post #2

The main reason for not using causal inference is not because data scientists don’t know about the different approaches or can’t imagine something equivalent (a lot of reinvention); forecasting is one of the most common tasks, after all. The main reason is that they generally work for software companies where it’s easier and less susceptible to analyst influence to implement the suggested change and test it with a Ra…

Actually causal inference is also really hard to benchmark. My colleague started an effort to be actually able to reproduce and compare results. Also the algorithms often do not scale too well. Everytime we wanted to use this for real data it is just a little bit too much effort and the results are not conclusive because it is hard to verify huge graphs. My colleague e.g. wanted to apply it explain risk confounders i…

You don’t need to look at a graph at all though, right? There are plenty of tests that can help you identify factors that could be significantly affecting your distribution

Re: Causal inference as a blind spot of data scientists

#14

Post by mjburgess »

tomrod wrote: post #11
mjburgess wrote: post #10

Earlier quoted context omitted.

To be clear, the mechanism for checking ML doesn't really check ML. There's really little value in a confidence interval conditional on the same experimental conditions that produced the dataset on which the model is trained. I'd often say it's actively harmful, since it's mostly misleading. Insofar as causal inference has no such 'check', its because there never was any. Casual inference is about dispelling that ill…

> Insofar as causal inference has no such 'check', its because there never was any. Casual inference is about dispelling that illusion. Aye, and that's the issue I'm trying to understand. How to know if model 1 or model 2 is more "real" or, for my lack of a better term, more useful and reflective of reality? We can focus on a particular philosophical point, like parsimony / Occam's razor, but as far as I can tell tha…

In general, you can't, and most of reality isnt knowable. That's a problem with reality, and us.

I'd take a bayesian approach across an ensemble of models based on the risk of each being right/wrong.

Consider whether Drug A causes or cures cancer. If there's some circumstantial evidence of it causing cancer at rate X in population Y with risk factors Z -- and otherwise broad circumstial evidence of it curing at rate A in pop B with features C...

then what? Then create various scenarios under these (likely contradictory) assumptions. Formulate an appropriate risk. Derive some implied policies.

This is the reality of how almost all actual decisions are made in life, and necessarily so.

The real danger is when ML is used to replace that, and you end up with extremely fragile systems that automate actions of unknown risk -- on the basis they were "99.99%", "accurate", ie., considered uncontrolled experimental condition E1 and not E2...10_0000 which actually occur

Re: Causal inference as a blind spot of data scientists

#15

Post by kjkjadksj »

tomrod wrote: post #11
mjburgess wrote: post #10

Earlier quoted context omitted.

To be clear, the mechanism for checking ML doesn't really check ML. There's really little value in a confidence interval conditional on the same experimental conditions that produced the dataset on which the model is trained. I'd often say it's actively harmful, since it's mostly misleading. Insofar as causal inference has no such 'check', its because there never was any. Casual inference is about dispelling that ill…

> Insofar as causal inference has no such 'check', its because there never was any. Casual inference is about dispelling that illusion. Aye, and that's the issue I'm trying to understand. How to know if model 1 or model 2 is more "real" or, for my lack of a better term, more useful and reflective of reality? We can focus on a particular philosophical point, like parsimony / Occam's razor, but as far as I can tell tha…

There is though. You run two linear models you get numbers back that inform how well these different models are fitting to the data.

Re: Causal inference as a blind spot of data scientists

#16

Post by rcthompson »

mwexler wrote: post #9

I 100% agree with this blind spot. Most data science coursework avoids the very thing making it a science: the explanation of what change causes what effect. I've been surprised that year after year, programs at so many "Schools of Data Science" keep gliding over this area, perhaps alluding to it in an early stats course if at all. It's an important part of validating that your data-driven output or decision is actua…

I realize this is nitpicking a minor point in your comment, but I don't agree with your characterization of RCTs in medical research as being primarily constrained by laws and regulations. Any time I've discussed research on human subjects with doctors doing that research, the discussion of what is and is not an acceptable experiment has always been primarily driven by the risks of harm to the people involved in the study. Any time the law comes up, it's usually because the law requires an RCT in a specific setting, as opposed to preventing it (e.g. drug trials). (Of course in the setting of starting a company based on some medical product, the situation may be quite different.)

Biologists, if not data scientists, are used to considering indirect evidence for causality. It's why we sometimes accept studies performed in other organisms as evidence for biology in humans; it's why we sometimes accept research performed on post-mortem human tissue as being representative of the biology of living humans; to name but a few examples. A big part of a compelling high-impact biology (or bioinformatics) paper is often the innovative ways that one comes up to show causality when a direct RCT is not feasible, and papers are frequently rejected because they don't to the follow-up experiments required to show causality.

Re: Causal inference as a blind spot of data scientists

#17

Post by nyssos »

paulpauper wrote: post #5

It's not that hard. if the causality cannot make sense logically or plausibly, then you can reasonably reject it . no reasonable person would ever get the umbrella puddles thing confused.

> It's not that hard. if the causality cannot make sense logically or plausibly, then you can reasonably reject it . no reasonable person would ever get the umbrella puddles thing confused.

It's an illustrative example, taking it literally is missing the point. The reason you know it doesn't make sense for umbrellas to cause rain is that you already have an applicable causal model. The situations where you need to do causal inference are exactly those where you don't, and can't just rely on "reasonableness" or "plausibility".

Re: Causal inference as a blind spot of data scientists

#18

Post by nyssos »

kjkjadksj wrote: post #15
tomrod wrote: post #11

Earlier quoted context omitted.

> Insofar as causal inference has no such 'check', its because there never was any. Casual inference is about dispelling that illusion. Aye, and that's the issue I'm trying to understand. How to know if model 1 or model 2 is more "real" or, for my lack of a better term, more useful and reflective of reality? We can focus on a particular philosophical point, like parsimony / Occam's razor, but as far as I can tell tha…

There is though. You run two linear models you get numbers back that inform how well these different models are fitting to the data.

This isn't a quality of fit issue (and even if it were, linear models are not always sufficient). The problem is that different causal structures can entail the same set of correlations, which makes them impossible to distinguish through observation alone.

Re: Causal inference as a blind spot of data scientists

#19

Post by mikpanko »

An important topic. Today most tech companies worship a/b experiments as the main way of being data-driven and bringing causality into decision-making. It deserves to be the gold standard.

However, most experiments are usually expensive: they require investing in building the feature in question and then collecting data for 1-4 weeks before being certain of the effects (plus there are long-term ones to worry about). Some companies report that fewer than 50% of their experiments prove truly impactful (my experience as well). That’s why only a small number of business decisions are made using experiments today.

Observational causal inference offers another approach, trading off full confidence in causality with speed and cost. It was pretty hard to run correctly so far, so it is not widely adopted. We are working on changing that with Motif Analytics and wrote a post with an in depth exploration of the problem: https://www.motifanalytics.com/blog/bringing-more-causality-... .

Re: Causal inference as a blind spot of data scientists

#20

Post by tomrod »

nyssos wrote: post #18
kjkjadksj wrote: post #15

Earlier quoted context omitted.

There is though. You run two linear models you get numbers back that inform how well these different models are fitting to the data.

This isn't a quality of fit issue (and even if it were, linear models are not always sufficient). The problem is that different causal structures can entail the same set of correlations, which makes them impossible to distinguish through observation alone.

Grandparent commenter here -- I'm glad I sufficiently communicated my concern, I feel like you and mjburgess have nailed it. Fit metrics alone aren't sufficient to determine an appropriate model use (even ignoring the issues of p-hacking an other ills).
Post reply on HN