For all of the examples on Wikipedia, it seems like there was some confounding extra variable that was missed. I wonder if anybody knows of a case where it just sort of happened randomly, with no big underlying cause? Or maybe I’m thinking of it wrong and this is impossible.
Simpson's paradox
71–80 of 111 posts
Re: Simpson's paradox
#72I absolutely love the Ellenberg quote: > Mathematician Jordan Ellenberg argues that Simpson's paradox is misnamed as "there's no contradiction involved, just two different ways to think about the same data" and suggests that its lesson "isn't really to tell us which viewpoint to take but to insist that we keep both the parts and the whole in mind at once." Keeping multiple possibilities in mind at once was what allow…
Isn't this a bit of a misunderstanding on their part on the meaning of the word "paradox"? The fact that they're called paradoxes is that they go against initial intuition and _seem_ contradictory, not that they necessarily are. If anything, I'd guess that most of the named paradoxes turn out to not actually be contradictory because when something seems incorrect and actually is incorrect, it's a lot less likely to be interesting enough to give a name.
Re: Simpson's paradox
#73Earlier quoted context omitted.
It’s actually surprisingly common. You can even find it in “classical” toy datasets like Iris: https://github.com/DataForScience/Causality/blob/master/1.2%...
Covid vaccination rates and deaths were rather famously subject to it. E.g. some combination of stats like “most covid deaths were vaccinated individuals”, “vaccination reduces death rate”, and “population segment with lowest vaccination rates has lowest covid death rates.” were all true at the same time.
Re: Simpson's paradox
#74Upon first glance I assumed it was the Simpson's episode about Mr. Burns having "a vast range of diseases so great in fact that they cancel each other out," explained here: https://simpsons.fandom.com/wiki/Three_Stooges_Syndrome Surprised at the similarity but of course that was probably on purpose by the genius Simpsons writers of the late 90's.
[0] https://www.independent.co.uk/arts-entertainment/tv/news/the...
Re: Simpson's paradox
#75What is the deal with posting (seemingly) random links to Wikipedia articles lately without context? Some of them are interesting, but most(?) of them come without details. Please provide me with some context.
Things hackers would find interesting. I think they have always been posted as long as I've been here.
There's been a huge uptick in submissions of random Wikipedia articles where there is no context provided with them.
There are maybe two or three in the last day that have anything beyond the title included. https://news.ycombinator.com/from?site=wikipedia.org
Re: Simpson's paradox
#76I was reading the example of UC Berkely appearing to have gender bias in the admissions and read the following: “it showed that women tended to apply to more competitive departments with lower rates of admission, even among qualified applicants (such as in the English department), whereas men tended to apply to less competitive departments with higher rates of admission (such as in the engineering department)” That’s…
The data is for application to graduate programs. There is a ton more funding for engineering, and many/most students going for a PhD in engineering don't pay for it. There's very little funding in the humanities, and most students are not willing to pay high costs for a PhD in the humanities, so the department tightly restricts admission.
As a result, it's easier to get into an engineering PhD program - as long as you are competent enough.
I had a friend who was a fellow engineering student. He became disillusioned and wanted to go into journalism. He applied to transfer to the Communications program at the university and told me how competitive it was - they admit less than 10 people per year. He did not get in.
[1] Obvious if you've spent a lot of time in grad school.
Re: Simpson's paradox
#77Earlier quoted context omitted.
Things hackers would find interesting. I think they have always been posted as long as I've been here.
There is no context provided with any of these links. I'm not making any claims about whether or not they have interesting content. There's been a huge uptick in submissions of random Wikipedia articles where there is no context provided with them. There are maybe two or three in the last day that have anything beyond the title included. https://news.ycombinator.com/from?site=wikipedia.org
Strange comment. There doesn't need to be.
Re: Simpson's paradox
#78Earlier quoted context omitted.
It is, but usually the meme misrepresents induced demand. While I don't like cars and we should focus on other infrastructure, adding a lane does help. It does not reduce congestion, but it does now serve more people at this same current congestion level. And those people have come from somewhere. Sometimes from public transport, which isn't really good, but sometimes from some backwater road.
The bigger problem with induced demand is that it's often poor ROI to add that lane where the demand is highest. That is, imagine you have a big city. You can add capacity for 1m extra people to travel to the city centre, where there's lots of congestion. Or you find ways to induce demand around the other limits of town, even town current demand is low there. Odds are you'll pick the first, because it's "obvious" and…
It's obvious at the supermarket: what goes faster, a single cashier processing four short lanes of 10 people with round robin, or two cashiers processing a single lane with 40 people?
Is the city center able to process 1m extra people? If not, it doesn't matter how many lanes you build.
Re: Simpson's paradox
#79https://en.wikipedia.org/wiki/Berkson%27s_paradox is also one to be aware of. There are lots of ways for error to creep in when populations are created in a biased way. These two effects explain a lot of the stupid decisions that come out of "data driven" processes. It is common for data to suggest the opposite of the truth.
> It is common for data to suggest the opposite of the truth. Actually, I think the best takeaway from phenomena like these is that just doing statistics on a set of data can't tell you "the truth". If you don't understand the actual causal factors in play, your knowledge is very limited, no matter how much data you have or how many different ways you slice the statistics. For example, in the UC Berkeley case describ…
Re: Simpson's paradox
#80I absolutely love the Ellenberg quote: > Mathematician Jordan Ellenberg argues that Simpson's paradox is misnamed as "there's no contradiction involved, just two different ways to think about the same data" and suggests that its lesson "isn't really to tell us which viewpoint to take but to insist that we keep both the parts and the whole in mind at once." Keeping multiple possibilities in mind at once was what allow…
Sorry to nitpick, but "light was made of discrete units that weighed very little and were moving very fast" is not really correct. First of all, light has exactly zero weight (only a massless particle can travel at exactly the speed of light, and at no other speed for that matter). Secondly, you're leaving out the wave/particle duality of light, which sort of reminds the Simpson's paradox description of "just two dif…