A Philosopher Reviews Judea Pearl's “The Book of Why” • Hacker News Board

Re: A Philosopher Reviews Judea Pearl's “The Book of Why”

#41

Post by fluentmundo » Tue, Sep 10, 2019, 4:01 PM UTC

ncmncm wrote: post #39

fluentmundo wrote: post #35

Earlier quoted context omitted.

A nice point. But this is not a criticism of the RCT so much as a criticism of how its results are used or interpreted.

It is a criticism of the notion of RCT as the unimpeachable "gold standard" of evidence. The limits of a tool are the most important thing to learn about it, and, for RCT, few can be bothered.

But it's not a limitation of the tool (the tool does provide gold-standard evidence); it's the stupidity of the researcher using the tool. (The tool works perfectly well and does, in fact, constitute the gold standard of causal evidence.) In your depression example, it's the stupidity of a researcher who fails to consider why a 1-in-6 success might be significant, or fails to consider that the umbrella disease "depression" can be multiply realized by different physiological mechanisms, which in turn require different treatments.

Your beef isn't with RCTs, or with the notion of RCT as evidence. Your beef is with researchers who don't know how to think.

Your complaint is like saying, we should stop considering hammers to be the gold-standard of nail-hitters because some idiots use them to try to turn screws.

Re: A Philosopher Reviews Judea Pearl's “The Book of Why”

#42

Post by ncmncm » Wed, Sep 11, 2019, 6:58 AM UTC

fluentmundo wrote: post #41

ncmncm wrote: post #39

Earlier quoted context omitted.

It is a criticism of the notion of RCT as the unimpeachable "gold standard" of evidence. The limits of a tool are the most important thing to learn about it, and, for RCT, few can be bothered.

But it's not a limitation of the tool (the tool does provide gold-standard evidence); it's the stupidity of the researcher using the tool. (The tool works perfectly well and does, in fact, constitute the gold standard of causal evidence.) In your depression example, it's the stupidity of a researcher who fails to consider why a 1-in-6 success might be significant, or fails to consider that the umbrella disease "depre…

My beef is with the public and policymakers convinced they have no need to understand the method's limitations, "because it's the gold standard", thus never misleading.

Re: A Philosopher Reviews Judea Pearl's “The Book of Why”

#43

Post by fluentmundo » Tue, Sep 17, 2019, 1:46 PM UTC

conjectures wrote: post #38

fluentmundo wrote: post #36

Earlier quoted context omitted.

>> Randomisation is a major part of traditional stats, and it is inherently a causal hypothesis Yes, randomization is central to classical statistics, but no, it is not inherently causal. Drawing a random sample from a bivariate distribution (X,Y) is key to doing a lot (though not all) of classical statistical inference (think of estimating slopes in regression), but the randomization does not imply anything about th…

I'm afraid you're mistaken. Randomisation allows one to make the strong causal assumption that the treatment regime allocation is unrelated to any of the other variables, observed or unobserved . Anyway, the section you're pointing to agrees with me. It just happens to be overlooked when they summarise...

No, I am not mistaken; you are confused. And your confusion is very pervasive in the technical community. You're not talking about randomization in general when you speak of a "treatment regime." You're talking about randomization in a causal experiment such as a randomized controlled trial. But "randomization" is a broader thing than randomization in a controlled experiment. The assumption of randomization is made for almost all classical statistical inference, which has nothing at all to do with causation.

Say you want to do basic linear regression: you want to estimate the slope for Y regressed on X. The most stringent form of inference works like this: you draw a random sample (X_i, Y_i), modeled as n independent and identically distributed realizations from the joint distribution (X,Y). Etc. This is certainly a stochastic model; we require randomization (or some approximation of it) to do inference. But it has nothing whatsoever to do with causality.