Live data from Hacker News

Simpson’s Paradox (2016)

forrestthewoods.com

51–60 of 84 posts

Re: Simpson’s Paradox (2016)

#51
post #42

Cool article. My knowledge of statistics is really rusty, but isn't this another way approaching the topic of "Bayesian Thinking"? If you think about the scenarios in the article from the standpoint of predicting any given outcome in advance, male vs. female and hard department vs. easy department should be treated as "priors". Or to put it another way, Bayesian thinking means asking the question "What is the chance…

This is actually a case that shows the limits of Bayesian thinking.

The power of probability is that it can work in two directions. You can use it to make predictions, from causes to effects, from past to future. Or you can use it to reason diagnostically, from effects to causes, like deducing what must have happened in the past to produce the current observation. Thinking probabilistically, these two cases are treated the same: they're both just conditioning on evidence, which is really elegant.

The problem is that when the two cases really need to be treated differently, probability can't distinguish between them. For example, asking about the probability of hypothetical situations, or predicting the results of interventions. You need to know which variables are causes and which are effects, but this is outside the scope of probability.

Simpson's paradox is something that only shows up when the variables involved have certain cause-effect structures. If you think in terms of these structures, it stops being counterintuitive.

Re: Simpson’s Paradox (2016)

#52
post #47
post #28

I'd like to say that the author has been reading The Book of Why , but it seems that he hasn't because he missed the punch line of the section on the paradox: you need a causal model to separate the two branches of the paradox. It's as easy to construct examples where the overall view is correct as it is so construct examples where the separate views are.

I'm unclear: what was the incorrect claim you're saying the author made?

The parent is not saying the author made an incorrect claim. They are saying that the parent did not continue their argument to arrive at a conclusion that someone else had, the conclusion that causal models are what tells you when you can combine datasets and when you can't.

Re: Simpson’s Paradox (2016)

#53
post #41

Earlier quoted context omitted.

This is less accurate, because not slicing data can also lead to bias.

Except the original statement didn't make any claim about "not slicing", so neither does mine.

Not slicing is nevertheless slicing. The trivial selection. Rush's song "If you choose not to decide you still have made a choice"

Re: Simpson’s Paradox (2016)

#54
post #42

Cool article. My knowledge of statistics is really rusty, but isn't this another way approaching the topic of "Bayesian Thinking"? If you think about the scenarios in the article from the standpoint of predicting any given outcome in advance, male vs. female and hard department vs. easy department should be treated as "priors". Or to put it another way, Bayesian thinking means asking the question "What is the chance…

This is more about knowing what the right question to ask is, which is trickier than expected. In the classic example, the people who brought the lawsuit asked “what are the odds of getting into Berkeley if you are a woman?” However, if people don’t apply to “Berkeley” but instead to “Berkeley’s College of Engineering”, then the right question is “what are the odds of getting into Berkeley’s college of engineering if you’re a woman”. The paradox is due to the fact that we expect the answers to be the same.

And all of this, of course, ignores sampling bias…

Re: Simpson’s Paradox (2016)

#55
post #47
post #28

I'd like to say that the author has been reading The Book of Why , but it seems that he hasn't because he missed the punch line of the section on the paradox: you need a causal model to separate the two branches of the paradox. It's as easy to construct examples where the overall view is correct as it is so construct examples where the separate views are.

I'm unclear: what was the incorrect claim you're saying the author made?

Simpson's paradox is about a data conflict between an overall view and a more specific view. For example, in the kidney stone scenario, say you find treatment A is more successful overall and treatment B is more successful in the specific view at both treating small stones and treating big stones when broken down that way. The article indicates that the specific view is always correct so treatment B should be used in the future, whereas the commenter is saying that context is important to determine which treatment should be used.

Re: Simpson’s Paradox (2016)

#56
post #37

Earlier quoted context omitted.

The trouble is you can't fix large numbers of statistical confounders with more data because there is a limit for how many factors you can control for before the measurement error overwhelms the signal. To do statistical controls, you essentially sort the data by category, so that you're not just comparing black people with white people, you're comparing middle class 18 year old black female college applicants with c…

I hate to take “both sides” but in the absence of confounding by indication, you can often use propensity scoring within robust models to decrease these impacts. Mind you, the problem with non random and undetected sampling bias is that it can be subtle. See for example https://www.nytimes.com/2018/08/06/upshot/employer-wellness-...

Propensity scoring is a method of applying statistical controls. How does it address the issue of controls compounding measurement error?

Re: Simpson’s Paradox (2016)

#57
post #23

Judea Pearl’s explanations of this in terms of causality are the only way it really makes sense, in my view. https://ftp.cs.ucla.edu/pub/stat_ser/r414.pdf

Unless I'm misreading the take away is failure to appreciate graph/network theory is behind the Simpson paradox. And I think a lot of broken 20th century 'science'. Because theory was based on simplistic statistical analysis on processes with strong path dependence.

Re: Simpson’s Paradox (2016)

#59
post #47

Earlier quoted context omitted.

I'm unclear: what was the incorrect claim you're saying the author made?

Simpson's paradox is about a data conflict between an overall view and a more specific view. For example, in the kidney stone scenario, say you find treatment A is more successful overall and treatment B is more successful in the specific view at both treating small stones and treating big stones when broken down that way. The article indicates that the specific view is always correct so treatment B should be used in…

Exactly. With a causal model (which can be validated independently) you have a principled reason for choosing which variables to control for.
Post reply on HN