[deleted]
Causal inference as a blind spot of data scientists
61–70 of 106 posts
Re: Causal inference as a blind spot of data scientists
#62I’ve tried to understand causal inference several times and failed. Tutorials seem unnecessarily long winded. I wish authors would give simple, to the point examples. Say I have a simple table of outdoor temperatures and ice cream sales. What can the machinery of causal inference do for me in this situation? If it doesn’t apply here, what do I need to add to my dataset to make it appropriate for causal inference? Mor…
My understanding (that might be out of date) is that the tools are weak. Ideally you would have tabular data and it would give you a digraph for the causal structure between variables. You can try this but the tools don't work reliably yet. Otherwise everyone would use them.
Re: Causal inference as a blind spot of data scientists
#63I’ve tried to understand causal inference several times and failed. Tutorials seem unnecessarily long winded. I wish authors would give simple, to the point examples. Say I have a simple table of outdoor temperatures and ice cream sales. What can the machinery of causal inference do for me in this situation? If it doesn’t apply here, what do I need to add to my dataset to make it appropriate for causal inference? Mor…
My understanding (that might be out of date) is that the tools are weak. Ideally you would have tabular data and it would give you a digraph for the causal structure between variables. You can try this but the tools don't work reliably yet. Otherwise everyone would use them.
Agreed. Afaict, in practice, you setup your own casual graphs and test them. This seems very academic 1950s.
Interestingly, folks are finally doing more realistic experiments in the casual equiv of arch search, and genAI is giving these efforts a second wind. Still feels like at the toy stage or for academics & researchers with a lot of time on their hands, vs relevant for most data scientists.
I'm still on the sidelines, but keep checking in in case finally practical for our users..
Re: Causal inference as a blind spot of data scientists
#64I think in practice most of these techniques are useless (or worse, confer a false sense of precision) because they require so many nuanced judgement calls that they become little more than a way to launder biases.
Re: Causal inference as a blind spot of data scientists
#65Earlier quoted context omitted.
My understanding (that might be out of date) is that the tools are weak. Ideally you would have tabular data and it would give you a digraph for the causal structure between variables. You can try this but the tools don't work reliably yet. Otherwise everyone would use them.
Agreed. Afaict, in practice, you setup your own casual graphs and test them. This seems very academic 1950s. Interestingly, folks are finally doing more realistic experiments in the casual equiv of arch search, and genAI is giving these efforts a second wind. Still feels like at the toy stage or for academics & researchers with a lot of time on their hands, vs relevant for most data scientists. I'm still on the sidel…
Same here, I check in every year or so because it would be fantastic to have.
Re: Causal inference as a blind spot of data scientists
#66In Corporate and Medical data science fields, people begin to accept causal inference. It is difficult, as the subject is still in flux and under development. I am aware of three reputable causal inference frameworks: 1. Judea Pearl's framework, which dominates in CS and AI circles 2. Neyman-Rubin causal model: https://en.wikipedia.org/wiki/Rubin_causal_model 3. Structural equation modelling: https://en.wikipedia.org…
Social sciences haven't ignored causal inference. Perhaps it’s not everywhere you’d like to see it, but it’s common in quant papers, its the backbone of econometrics, and you’d probably have trouble finding a single top ranked PhD program which doesn’t provide at least cursory coverage of the methods.
Re: Causal inference as a blind spot of data scientists
#67I’ve tried to understand causal inference several times and failed. Tutorials seem unnecessarily long winded. I wish authors would give simple, to the point examples. Say I have a simple table of outdoor temperatures and ice cream sales. What can the machinery of causal inference do for me in this situation? If it doesn’t apply here, what do I need to add to my dataset to make it appropriate for causal inference? Mor…
It's pretty simple. You cannot infer casualty from observational data. No matter how sophisticated your statistical tools are.
You need to perform a properly controlled experiments to infer casualty. And even then it's hard.
Inferring casualty from observational data is cargo cult science.
Re: Causal inference as a blind spot of data scientists
#68> The DoubleML method is founded on machine learning modeling and consists of two key steps. First, we build a model that predicts the treatment variable based on the input variables . Then, we create a separate model that predicts the outcome variable using the same set of input variables . Subsequently, we calculate the residuals from the former model and regress them against the residuals from the latter model. An important feature of this method is its flexibility in accommodating non-linear models, which allows us to capture non-linear relationships — a distinctive advantage of this approach.
Just... don't do this. You're not going to be able to math your way to better conclusions. Make your model, make your plots, and use critical thinking to evaluate your results.
Re: Causal inference as a blind spot of data scientists
#69I’ve tried to understand causal inference several times and failed. Tutorials seem unnecessarily long winded. I wish authors would give simple, to the point examples. Say I have a simple table of outdoor temperatures and ice cream sales. What can the machinery of causal inference do for me in this situation? If it doesn’t apply here, what do I need to add to my dataset to make it appropriate for causal inference? Mor…
> Say I have a simple table of outdoor temperatures and ice cream sales. What can the machinery of causal inference do for me in this situation?
Not much. Causal inference works over networks of variables, specifically a DAG. But usually you know more than one variable association, so this is more an issue of pedagogy than the tool itself.
Probably the shortest, most persuasive example I can give you is a logical resolution to Simpson's Paradox: when the correlation between two variables can change depending on whether you consider a third variable or not.
The classic example is gender discrimination in college admissions. When looking at admissions rates across the entire university, women are less likely to be accepted than men. But when (in this example) you break that down into departments, every department favors women over men. This is a paradoxical contradiction, and worrying in that your science is only as good as the dimensions your data captures. Worse, the data offers no clean way to say which is the correct answer: the aggregate or the total. Statisticians stumbled for a long while on this, and it's kind of wild that we were able to declare smoking causes cancer without a resolution to this.
Pearl wrote a paper on how bayesian approaches resolve the paradox[1], but it does presume familiarity with terms like "colliders," "backdoor criterion" and "do-calculus." His main point is that causal inference techniques give us the language and tools to resolve the paradox that frequentist approaches do not.
Re: Causal inference as a blind spot of data scientists
#70The Atlantic/American Causal Inference Conference (ACIC) hosts a data challenge every year, I think. Useful to see many different methods compared on simulated data.
Does anyone know similar challenges/competitions?
ACIC links to years I could find:
- 2016: https://arxiv.org/abs/1707.02641
- 2017: https://arxiv.org/abs/1905.09515
- 2019: https://sites.google.com/view/acic2019datachallenge/data-cha...