Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simpson’s Paradox (2016)

forrestthewoods.com

Post reply on HN
Print view View original

11–20 of 84 posts

Re: Simpson’s Paradox (2016)

#11

Post by TicklishTiger »

sopooneo wrote: post #7
TicklishTiger wrote: post #4

That is not a paradox. It's just the fact that a theory about something might not hold when you take a closer look at that something. In the articles example, the admission rates of a university seemed to indicate that there is a bias against women. Zooming in and looking at the admission rates of the individual departments seem to indicate that there is a bias against men. The article makes it sound like the first t…

I don't know, but at some point, aren't we just running up against the definition of "probability"?

Probably.

Re: Simpson’s Paradox (2016)

#12

Post by knappa »

The sex-discrimination lawsuit against UC Berkley seems to be a kind of academic urban myth; the administration was apparently afraid of such a lawsuit and the study was done in response to those administrative fears.

Re: Simpson’s Paradox (2016)

#13

Post by esquire_900 »

This is the exact feeling I've been having for years, nicely described in an easy to understand language. At least in data science and (god forbid) behavioral psychology, you can answer any question any way you like - statistically valid - by slightly shifting the level of focus (as described here), definitions or angle of attack. The more data, the easier.

Thanks for putting it in such a clear way :)

Re: Simpson’s Paradox (2016)

#16

Post by n4r9 »

TicklishTiger wrote: post #4

That is not a paradox. It's just the fact that a theory about something might not hold when you take a closer look at that something. In the articles example, the admission rates of a university seemed to indicate that there is a bias against women. Zooming in and looking at the admission rates of the individual departments seem to indicate that there is a bias against men. The article makes it sound like the first t…

It is a paradox. In common usage, a paradox is an apparent absurdity which nevertheless holds up upon deeper investigation. In this case the apparent absurdity is e.g. "Treatment A is better at treating kidney stones despite performing worse in both trials".

Sometimes the word paradox has a slightly different meaning. For example, Russell's paradox in mathematics is the opposite; it takes something apparently well-founded and shows that it is absurd.

Re: Simpson’s Paradox (2016)

#17

Post by geocar »

TicklishTiger wrote: post #4

That is not a paradox. It's just the fact that a theory about something might not hold when you take a closer look at that something. In the articles example, the admission rates of a university seemed to indicate that there is a bias against women. Zooming in and looking at the admission rates of the individual departments seem to indicate that there is a bias against men. The article makes it sound like the first t…

That a pair of attributes doesn't necessarily exhibit independence within the universe at large, even if it exhibits independence within each sub-universe is a powerful observation, and it's a troubling one to anyone who has attempted to design a sales and marketing strategy, a drug trial, or frameworks to encourage social equality: To have it suggested I can say nothing less about these thousand students other than a thousand different things, just sounds so absurd, and yet here it is true.

Sometimes people use the term "paradox" simply to a contradictory statement which upon investigation turns out to be true. In that way, "Simpson's Paradox" is absolutely a paradox.

Re: Simpson’s Paradox (2016)

#18

Post by srean »

Simpson's Paradox is one of the many phenomena that shows how different applied ML is from regular software engineering. Another one is feedback loops between decomposed subproblems.

In ML encapsulation, shielding away of inner details often does not work. One needs to know what is happening on the other side of the abstraction boundary. This is a problem for managers and PM coning to ML from a purely software engineering background. They are used to encapsulation and decomposition serving them well and they expect the same.

Re: Simpson’s Paradox (2016)

#19

Post by AnthonyMouse »

rwilson4 wrote: post #8
sopooneo wrote: post #5

In simples case at least, such as with the kidney stones, can we reduce our risk of reaching wrong conclusions by increasing our sample size of patients and randomizing which receive each treatment?

Yes absolutely! Random assigment along with statistical power and significance considerations does indeed allow one to draw causal conclusions. It’s the gold standard for causal inference.

> Yes absolutely!

The problem with these cases is generally that people want to use data that didn't come from a controlled experiment to begin with. You have a nice, fat data set of all the people who have been treated for kidney stones -- you could never afford to do a controlled experiment at that scale. But because the treatments weren't randomized (and neither was anything else), the conclusions are erroneous.

This has been a huge problem in social sciences, where you can't do the controlled experiment at all, even at a smaller scale, because there is no way to randomize the choices individuals make. All you can do is try to control for the divergence statistically -- but there isn't one confounder in real data, there are thousands or more, and each one you want to control for multiplies the measurement error (because the measurement error in the primary factor combines with the measurement error in the control factor).

Post reply on HN