Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simpson's Paradox

vudlab.com

Post reply on HN
Print view View original

41–50 of 52 posts

Re: Simpson's Paradox

#41

Post by chrisbennet »

maxerickson wrote: post #5
tlb wrote: post #4

The Omitted Variable Problem is part of my mental framework that causes me to not believe most epidemiological studies, especially ones that confirm a popular belief. The almost universally omitted variable is health-consciousness. Some people are health-conscious and some aren't. People who are health-conscious do a whole bunch of things, some of which help (like exercise, sleep well, eat moderately). They also do t…

If it was a huge deal, wouldn't epidemiologists be slobbering over the statistical power they would get from using it? That is, if they can control for it, they can make assertions about smaller effects.

In his article "Science pseudoscience nutritional epidemiology and meat" http://garytaubes.com/2012/03/science-pseudoscience-nutritio... Gary Taubes discusses how epidemiological studies can fail to account for something called the "compliance effect" - the missing variable.

It was an eye opener for me.

Re: Simpson's Paradox

#42

Post by maxerickson »

chrisbennet wrote: post #41
maxerickson wrote: post #5

Earlier quoted context omitted.

If it was a huge deal, wouldn't epidemiologists be slobbering over the statistical power they would get from using it? That is, if they can control for it, they can make assertions about smaller effects.

In his article "Science pseudoscience nutritional epidemiology and meat" http://garytaubes.com/2012/03/science-pseudoscience-nutritio... Gary Taubes discusses how epidemiological studies can fail to account for something called the "compliance effect" - the missing variable. It was an eye opener for me.

He quotes a textbook chapter in his conclusion. So the idea that you have to at least be careful with your statistics is orthodoxy, not some missing component of the field.

(I realize that doesn't make any guarantees about common practices, but it is maybe a little capricious to repudiate the entire field)

Re: Simpson's Paradox

#43

Post by dbecker »

defen wrote: post #10
rthomas6 wrote: post #8

Earlier quoted context omitted.

Surely some studies account for this? If you look for people who do X and people who don't and just analyze their lives, yes this problem is likely to exist. But if you take two randomized samples of a the population and say to group A, "do X," and to group B "don't do X," you have an effective control group. At least I think so. Don't some dietary studies even provide the participants with custom food regimens to tr…

You're describing a randomized controlled trial and not an epidemiological study.

You are mistakenly confounding that epidemiological and observational.

Epidemiology studies can be randomized or observational.

Re: Simpson's Paradox

#44

Post by wnewman »

maxander wrote: post #29
gwern wrote: post #18

Earlier quoted context omitted.

> If it was a huge deal, wouldn't epidemiologists be slobbering over the statistical power they would get from using it? Why would you want to do that? You can build a nice career on publishing (spurious) associations. As long as relevant variables are omitted, no epidemiologist need ever be unemployed.

That may get you written about in popular-science magazines, but your work will have a low impact factor- this is largely measured in how many of your fellow researchers cite your work, and its crucial to an academic researcher's career. If your colleagues can easily see that your statistical analysis is flawed, they won't waste their own time doing work that builds off of yours.

It is not necessarily true that just because they are called scientists, real people with real motives, funded by organizations of real people with real motives, operating in a political regime controlled by real people with real motives, will act that way. (By "that way" I mean "your work will have a low impact factor- this is largely measured in how many of your fellow researchers cite your work" and "If your colleagues can easily see that your statistical analysis is flawed, they won't waste their own time doing work that builds off of yours.")

See e.g. Chan and Boliver "The Grandparents Effect in Social Mobility", http://asr.sagepub.com/content/78/4/662 (HT http://www.arnoldkling.com/blog/a-grandparent-effect/) and many of the works that it cites. There seems to be a sizable mutually-citing "scientific" literature devoted to studying correlation in biological descent while dogmatically not controlling for ordinary DNA-based inheritance of psychological traits from parent to child. (Indeed, not even controlling for DNA-based inheritance of physical traits. Health and height and attractiveness are correlated with income, and are significantly physically heritable.) By not controlling for the properties which are transmitted by sperm meeting egg, your can find all sorts of fascinating possibilities to investigate to explain the correlation. (E.g. Chan and Boliver carefully note that "well-connected grandparents could also use their social contacts to help grandchildren with job searches." Good, good, carefully investigate all the possibilities.)

Re: Simpson's Paradox

#45

Post by publicfig »

teamonkey wrote: post #20
publicfig wrote: post #13

Earlier quoted context omitted.

I feel as thought both graphs with purple and green lines are accurately explained. Were you trying to skim through the article or were you still left confused after reading it? The top one is explained in the accompanying text and the bottom one is explained with a labeled x and y axis and relies on the information provided above.

If the article describes the subject adequately, what is the point of the graphic? I found the interactive graphic more confusing than the description. Compare and contrast to the clarity of their central limit theorem demonstation [1] vs its wikipedia page [2]. [1] http://blog.vctr.me/posts/central-limit-theorem.html [2] http://en.wikipedia.org/wiki/Central_limit_theorem

By that, I meant that the article described what the graph was representing, not that the article described the information presented adequately. Also, I absolutely disagree with the assumption you made that graphs shouldn't be present if the information is presented in the article. It's good to provide visual demonstrations when applicable in order to help clarify subjects.

Re: Simpson's Paradox

#46

Post by tel »

jasonwatkinspdx wrote: post #27

Judea Pearl has formalized a resolution to Simpson's Paradox: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.34.... An oversimplification of his idea would be to say that given assumptions about the causal independence of variables, it becomes clear which way you should group the data. Although it's not always possible to make these independence assumptions, much of the time they are obvious and uncontrover…

That's a great reference. I'm going to use it from now on too.

Re: Simpson's Paradox

#47

Post by defen »

dbecker wrote: post #43
defen wrote: post #10

Earlier quoted context omitted.

You're describing a randomized controlled trial and not an epidemiological study.

You are mistakenly confounding that epidemiological and observational. Epidemiology studies can be randomized or observational.

Interesting...what would be an example of a randomized epidemiological study?

Re: Simpson's Paradox

#48

Post by B-Con »

tlb wrote: post #4

The Omitted Variable Problem is part of my mental framework that causes me to not believe most epidemiological studies, especially ones that confirm a popular belief. The almost universally omitted variable is health-consciousness. Some people are health-conscious and some aren't. People who are health-conscious do a whole bunch of things, some of which help (like exercise, sleep well, eat moderately). They also do t…

> What Simpson's paradox is not

> Every ommitted variable problem

Not quite the same thing as the OP (not sure if you were implying it was). Regardless, OVP is still a great thing to have in your mental framework.

Re: Simpson's Paradox

#49

Post by gwern »

maxander wrote: post #29
gwern wrote: post #18

Earlier quoted context omitted.

> If it was a huge deal, wouldn't epidemiologists be slobbering over the statistical power they would get from using it? Why would you want to do that? You can build a nice career on publishing (spurious) associations. As long as relevant variables are omitted, no epidemiologist need ever be unemployed.

That may get you written about in popular-science magazines, but your work will have a low impact factor- this is largely measured in how many of your fellow researchers cite your work, and its crucial to an academic researcher's career. If your colleagues can easily see that your statistical analysis is flawed, they won't waste their own time doing work that builds off of yours.

> but your work will have a low impact factor- this is largely measured in how many of your fellow researchers cite your work, and its crucial to an academic researcher's career.

Nope! This is how we might like to to work, but cool associations will get written up forever, long after they have been refuted by randomized experiments. (Heck, papers can outright be retracted and still get cites.)

Hahaha - did I say refuted by randomized experiments? Like that ever happens to more than a tiny fraction of epidemiology correlations... No, one's career is quite safe. No one ever lost tenure because the correlations they built their career on turned out to be lame.

> If your colleagues can easily see that your statistical analysis is flawed, they won't waste their own time doing work that builds off of yours.

No, the point here is that your work can be immaculate and still never reflect causality. How are you going to collect data on every lurking variable? You can't, of course.

Re: Simpson's Paradox

#50

Post by jasonwatkinspdx »

tel wrote: post #46
jasonwatkinspdx wrote: post #27

Judea Pearl has formalized a resolution to Simpson's Paradox: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.34.... An oversimplification of his idea would be to say that given assumptions about the causal independence of variables, it becomes clear which way you should group the data. Although it's not always possible to make these independence assumptions, much of the time they are obvious and uncontrover…

That's a great reference. I'm going to use it from now on too.

I won't claim to have fully internalized all of it, but Judea Pearl's book Causality is incredible.
Post reply on HN