Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simpson's Paradox

vudlab.com

Post reply on HN
Print view View original

11–20 of 52 posts

Re: Simpson's Paradox

#11

Post by spaux »

rthomas6 wrote: post #8
tlb wrote: post #4

The Omitted Variable Problem is part of my mental framework that causes me to not believe most epidemiological studies, especially ones that confirm a popular belief. The almost universally omitted variable is health-consciousness. Some people are health-conscious and some aren't. People who are health-conscious do a whole bunch of things, some of which help (like exercise, sleep well, eat moderately). They also do t…

Surely some studies account for this? If you look for people who do X and people who don't and just analyze their lives, yes this problem is likely to exist. But if you take two randomized samples of a the population and say to group A, "do X," and to group B "don't do X," you have an effective control group. At least I think so. Don't some dietary studies even provide the participants with custom food regimens to tr…

What you're describing is the "as long as you omit health-consciousness as a variable" part of his comment.

Re: Simpson's Paradox

#13

Post by publicfig »

teamonkey wrote: post #7

The point of an interactive illustration like this is that it should make it more intuitive and easier to understand than the core concept, not more confusing. What is the meaning of the green and purple lines on the graph, why do they have different gradients and why can't I adjust them? Why does the Simpson's Paradox apply sometimes and not others? Why are there so many bars and donut charts? What does the gray cir…

I feel as thought both graphs with purple and green lines are accurately explained. Were you trying to skim through the article or were you still left confused after reading it? The top one is explained in the accompanying text and the bottom one is explained with a labeled x and y axis and relies on the information provided above.

Re: Simpson's Paradox

#14

Post by trains »

It's nice visually but the text-based information could be more thorough - which would result in less links at the bottom.The more info contained on the page the better!

Re: Simpson's Paradox

#15

Post by jere »

teamonkey wrote: post #7

The point of an interactive illustration like this is that it should make it more intuitive and easier to understand than the core concept, not more confusing. What is the meaning of the green and purple lines on the graph, why do they have different gradients and why can't I adjust them? Why does the Simpson's Paradox apply sometimes and not others? Why are there so many bars and donut charts? What does the gray cir…

It took a little playing around to figure out some of the graphics.

The donuts indicate which group (men or women) was admitted more in each kind of department.

Play around with the sliders and you can see it toggle in the combined row, while simultaneously a yes/no answer updates after the question "Simpson's paradox?" It would really help if yes/no was highlighted. Basically, you can make the paradox appear by dropping the % of women applying to easy departments. You can't make the donut switch in easy or hard departments individually... that's the whole point: women have a slight advantage in each.

Honestly, I found the "Proper Pooling" section to have the most intuitive explanation. Combined, there is a bias. By department, there is no bias. The reason is clear: very few women applying to easier departments.

Re: Simpson's Paradox

#16

Post by jezebel6 »

teamonkey wrote: post #7

The point of an interactive illustration like this is that it should make it more intuitive and easier to understand than the core concept, not more confusing. What is the meaning of the green and purple lines on the graph, why do they have different gradients and why can't I adjust them? Why does the Simpson's Paradox apply sometimes and not others? Why are there so many bars and donut charts? What does the gray cir…

Agreed. The text is straightforward enough to understand, but the graphics are irritating and don't contribute much.

Re: Simpson's Paradox

#17

Post by alexPetrov »

Reading up on Simpson's Paradox again made me realize something: women retrieving custody more often than men appears to me to be a perfect example of Simpson's Paradox. Overall, women get custody more often than men, this is true, but if you consider only the cases where men actually asked for or attempted to retrieve custody, this is no longer true[0].

[0]: http://www.villainouscompany.com/vcblog/archives/2012/04/chi...

Re: Simpson's Paradox

#18

Post by gwern »

maxerickson wrote: post #5
tlb wrote: post #4

The Omitted Variable Problem is part of my mental framework that causes me to not believe most epidemiological studies, especially ones that confirm a popular belief. The almost universally omitted variable is health-consciousness. Some people are health-conscious and some aren't. People who are health-conscious do a whole bunch of things, some of which help (like exercise, sleep well, eat moderately). They also do t…

If it was a huge deal, wouldn't epidemiologists be slobbering over the statistical power they would get from using it? That is, if they can control for it, they can make assertions about smaller effects.

> If it was a huge deal, wouldn't epidemiologists be slobbering over the statistical power they would get from using it?

Why would you want to do that? You can build a nice career on publishing (spurious) associations. As long as relevant variables are omitted, no epidemiologist need ever be unemployed.

Re: Simpson's Paradox

#19

Post by marcosdumay »

maxerickson wrote: post #5
tlb wrote: post #4

The Omitted Variable Problem is part of my mental framework that causes me to not believe most epidemiological studies, especially ones that confirm a popular belief. The almost universally omitted variable is health-consciousness. Some people are health-conscious and some aren't. People who are health-conscious do a whole bunch of things, some of which help (like exercise, sleep well, eat moderately). They also do t…

If it was a huge deal, wouldn't epidemiologists be slobbering over the statistical power they would get from using it? That is, if they can control for it, they can make assertions about smaller effects.

Controling for hidden variables (the general case of this paradox) is a big deal for all science areas.

Re: Simpson's Paradox

#20

Post by teamonkey »

publicfig wrote: post #13
teamonkey wrote: post #7

The point of an interactive illustration like this is that it should make it more intuitive and easier to understand than the core concept, not more confusing. What is the meaning of the green and purple lines on the graph, why do they have different gradients and why can't I adjust them? Why does the Simpson's Paradox apply sometimes and not others? Why are there so many bars and donut charts? What does the gray cir…

I feel as thought both graphs with purple and green lines are accurately explained. Were you trying to skim through the article or were you still left confused after reading it? The top one is explained in the accompanying text and the bottom one is explained with a labeled x and y axis and relies on the information provided above.

If the article describes the subject adequately, what is the point of the graphic? I found the interactive graphic more confusing than the description.

Compare and contrast to the clarity of their central limit theorem demonstation [1] vs its wikipedia page [2].

[1] http://blog.vctr.me/posts/central-limit-theorem.html

[2] http://en.wikipedia.org/wiki/Central_limit_theorem

Post reply on HN