Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Causal inference as a blind spot of data scientists

dzidas.com

Post reply on HN
Print view View original

31–40 of 106 posts

Re: Causal inference as a blind spot of data scientists

#31

Post by mbowcut2 »

For what it’s worth, my undergraduate was in Economics with an emphasis in econometrics and this article touched on probably 80% of the curriculum.

The only problem is by the time I graduated I was somewhat disillusioned with most causal inference methods. It takes a perfect storm natural experiment to get any good results. Plus every 5 years a paper comes out that refutes all previous papers that use whatever method was in vogue at the time.

This article makes me want to get back into this type of thinking though. It’s refreshing after years of reading hand-wavy deep learning papers where SOTA is king and most theoretical thinking seems to occur post hoc, the day of the submission deadline.

Re: Causal inference as a blind spot of data scientists

#32

Post by gridland »

mbowcut2 wrote: post #31

For what it’s worth, my undergraduate was in Economics with an emphasis in econometrics and this article touched on probably 80% of the curriculum. The only problem is by the time I graduated I was somewhat disillusioned with most causal inference methods. It takes a perfect storm natural experiment to get any good results. Plus every 5 years a paper comes out that refutes all previous papers that use whatever method…

just use propensity scores + ipw and you have the same thing as a rct. :)

Re: Causal inference as a blind spot of data scientists

#33

Post by cubefox »

g42gregory wrote: post #22

In Corporate and Medical data science fields, people begin to accept causal inference. It is difficult, as the subject is still in flux and under development. I am aware of three reputable causal inference frameworks: 1. Judea Pearl's framework, which dominates in CS and AI circles 2. Neyman-Rubin causal model: https://en.wikipedia.org/wiki/Rubin_causal_model 3. Structural equation modelling: https://en.wikipedia.org…

> In Social Sciences, on the other hand, causal inference is being completely willfully ignored.

This conflicts with what the article says:

> Economists and social scientists were among the first to recognize the advantages of these emerging causal inference techniques and incorporated in their research.

Re: Causal inference as a blind spot of data scientists

#34

Post by hackernewds »

gridland wrote: post #32
mbowcut2 wrote: post #31

For what it’s worth, my undergraduate was in Economics with an emphasis in econometrics and this article touched on probably 80% of the curriculum. The only problem is by the time I graduated I was somewhat disillusioned with most causal inference methods. It takes a perfect storm natural experiment to get any good results. Plus every 5 years a paper comes out that refutes all previous papers that use whatever method…

just use propensity scores + ipw and you have the same thing as a rct. :)

not necessarily unless you skim over meaningful confounding factors :)

Re: Causal inference as a blind spot of data scientists

#35

Post by hackernewds »

mikpanko wrote: post #21
bertil wrote: post #2

The main reason for not using causal inference is not because data scientists don’t know about the different approaches or can’t imagine something equivalent (a lot of reinvention); forecasting is one of the most common tasks, after all. The main reason is that they generally work for software companies where it’s easier and less susceptible to analyst influence to implement the suggested change and test it with a Ra…

A/b experiments are definitely a gold standard as they provide true causality measurement (if implemented correctly). However, they are often expensive to run: need to implement the feature in question (which is less than 50% going to work) and then collect data for 1-4 weeks before being able to make the decision. As a result only a small number of business decisions today rely on a/b tests. Observational causal inf…

an expensive test is better than an expensive mistake :) within the scale of hundreds of decisions made with inherent bias of the product/biz/ops teams that direction misalignment can be catastrophic

Re: Causal inference as a blind spot of data scientists

#36

Post by mikpanko »

bertil wrote: post #24
mikpanko wrote: post #21

Earlier quoted context omitted.

A/b experiments are definitely a gold standard as they provide true causality measurement (if implemented correctly). However, they are often expensive to run: need to implement the feature in question (which is less than 50% going to work) and then collect data for 1-4 weeks before being able to make the decision. As a result only a small number of business decisions today rely on a/b tests. Observational causal inf…

Causal inference is useful, but it's neither quicker nor cheaper.

Agree that it is hard today. A person you might know is trying to prove that is doesn’t have to be: https://www.motifanalytics.com/blog/bringing-more-causality-... .

We’d love to chat more with you on the topic - feel free to hit Sean or me on LinkedIn.

Re: Causal inference as a blind spot of data scientists

#37

Post by esafak »

riedel wrote: post #7
bertil wrote: post #2

The main reason for not using causal inference is not because data scientists don’t know about the different approaches or can’t imagine something equivalent (a lot of reinvention); forecasting is one of the most common tasks, after all. The main reason is that they generally work for software companies where it’s easier and less susceptible to analyst influence to implement the suggested change and test it with a Ra…

Actually causal inference is also really hard to benchmark. My colleague started an effort to be actually able to reproduce and compare results. Also the algorithms often do not scale too well. Everytime we wanted to use this for real data it is just a little bit too much effort and the results are not conclusive because it is hard to verify huge graphs. My colleague e.g. wanted to apply it explain risk confounders i…

Go on, please. What definition, and algorithms with scaling problems?

Re: Causal inference as a blind spot of data scientists

#38

Post by esafak »

kjkjadksj wrote: post #13
riedel wrote: post #7

Earlier quoted context omitted.

Actually causal inference is also really hard to benchmark. My colleague started an effort to be actually able to reproduce and compare results. Also the algorithms often do not scale too well. Everytime we wanted to use this for real data it is just a little bit too much effort and the results are not conclusive because it is hard to verify huge graphs. My colleague e.g. wanted to apply it explain risk confounders i…

You don’t need to look at a graph at all though, right? There are plenty of tests that can help you identify factors that could be significantly affecting your distribution

You can only select among features that you have measured.

Re: Causal inference as a blind spot of data scientists

#39

Post by esafak »

bertil wrote: post #24
mikpanko wrote: post #21

Earlier quoted context omitted.

A/b experiments are definitely a gold standard as they provide true causality measurement (if implemented correctly). However, they are often expensive to run: need to implement the feature in question (which is less than 50% going to work) and then collect data for 1-4 weeks before being able to make the decision. As a result only a small number of business decisions today rely on a/b tests. Observational causal inf…

Causal inference is useful, but it's neither quicker nor cheaper.

It is likely to be cheaper and quicker to run a counterfactual test in the computer than in real life.

The question is how reliable it is.

Re: Causal inference as a blind spot of data scientists

#40

Post by qmsoqm »

Anon84 wrote: post #3

For a hands on introduction to Causality, I would recommend “Causal Inference in Python” by M. Facure https://amzn.to/46byWnl Well written and to the point. I also have a series of blog posts on the topic: https://github.com/DataForScience/Causality where I work through Pearls Primer: https://amzn.to/3gsFlkO

Thanks for the recommendation
Post reply on HN