Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Causal inference as a blind spot of data scientists

dzidas.com

Post reply on HN
Print view View original

101–106 of 106 posts

Re: Causal inference as a blind spot of data scientists

#101

Post by mmmmpancakes »

Trombone12 wrote: post #79
mmmmpancakes wrote: post #77

Earlier quoted context omitted.

Yeah, the only common theme I see in causal inference research is that every method and analysis eventually succumbs to a more thorough analysis that uncovers serious issues in the assumptions. Take for instance the running example of catholic schoolings effect on test scores used by the boook Counterfactuals and Causal Inference. Subsequent chapter re-treat this example with increasingly sophisticated techniques and…

> My lesson from this: outcomes causal inference is very dependent on assumptions and methodologies, of which the options are many. This seems like a natural feature of any sensitive method, not sure why this is something to complain about. If you want your model to always give the answer you expected you don't actually have to bother collecting data in the first place, just write the analysis the way pundits do.

> This seems like a natural feature of any sensitive method, not sure why this is something to complain about.

I am exactly complaining it is sensitive. If theres robust alternatives why would i put this in prod?

Re: Causal inference as a blind spot of data scientists

#102

Post by Trombone12 »

mmmmpancakes wrote: post #101
Trombone12 wrote: post #79

Earlier quoted context omitted.

> My lesson from this: outcomes causal inference is very dependent on assumptions and methodologies, of which the options are many. This seems like a natural feature of any sensitive method, not sure why this is something to complain about. If you want your model to always give the answer you expected you don't actually have to bother collecting data in the first place, just write the analysis the way pundits do.

> This seems like a natural feature of any sensitive method, not sure why this is something to complain about. I am exactly complaining it is sensitive. If theres robust alternatives why would i put this in prod?

Because you care about accuracy?

Re: Causal inference as a blind spot of data scientists

#104

Post by kjkjadksj »

nyssos wrote: post #18
kjkjadksj wrote: post #15

Earlier quoted context omitted.

There is though. You run two linear models you get numbers back that inform how well these different models are fitting to the data.

This isn't a quality of fit issue (and even if it were, linear models are not always sufficient). The problem is that different causal structures can entail the same set of correlations, which makes them impossible to distinguish through observation alone.

Such errors can be estimated with other models

https://pubmed.ncbi.nlm.nih.gov/22364439/

Re: Causal inference as a blind spot of data scientists

#105

Post by mbowcut2 »

Trombone12 wrote: post #102
mmmmpancakes wrote: post #101

Earlier quoted context omitted.

> This seems like a natural feature of any sensitive method, not sure why this is something to complain about. I am exactly complaining it is sensitive. If theres robust alternatives why would i put this in prod?

Because you care about accuracy?

But really, how accurate are your results if they depend on strong assumptions about your data?

Re: Causal inference as a blind spot of data scientists

#106

Post by mmmmpancakes »

Trombone12 wrote: post #102
mmmmpancakes wrote: post #101

Earlier quoted context omitted.

> This seems like a natural feature of any sensitive method, not sure why this is something to complain about. I am exactly complaining it is sensitive. If theres robust alternatives why would i put this in prod?

Because you care about accuracy?

Because with real world data like in production in tech there are so many factors to account for. Brittle methods are more susceptible to unexpected changes in the data or unexpected ways in which complex assumptions abut the data fail.
Post reply on HN