What is the deal with posting (seemingly) random links to Wikipedia articles lately without context?
Some of them are interesting, but most(?) of them come without details.
Please provide me with some context.
Simpson's paradox
21–30 of 111 posts
Re: Simpson's paradox
#22I once encountered this in the real world as a data analyst a long time ago. I was working at an e-commerce company, called The Hut Group, and the whole year our marketing team had been saying our marketing cost of goods sold (the percentage of our revenue we needed to spend on marketing) had been declining across every product category. But at year end, the execs were shocked to realize that our cost of goods sold h…
Pretty much every dataset I work with as an SRE is full of these paradoxes. One classic published example comes from Google:
A network engineer took a trip to Indonesia or something (can't find the citation to confirm the exact tale), noticed the service was slow, and when asking around everyone said "that's how its always been." Basically the local cellular networks are slow and off island fiber connects are saturated. Back at the office they decide to attack the problem by optimizing payload sizes. Does the work, reducing download sizes by half, and ships it. Latency metrics? Average and p95 latency actually increased after shipping the work to production.
How does an objectively good change make things worse? Well, the service had improved for those customers so much that they used it a lot more. Even with the lighter demand on bandwidth the network latency to the datacenter was worse than typical US customers, so as more of these people realized the service sucked way less, they used it more and drove the numbers up.
I have tons of these examples where a data team looks at a particular slice of request telemetry, and comes to a wrong conclusion because they didn't model enough of the system, or controlled for the wrong (or too many) variables. The worst ones the cyclic finger pointing situations that Simpson's paradox can produce: App developers blaming a regression on the server side component while the server team blames the app team, often because the server and app release schedules accidentally aligned too well. In this case we have canary data to exonerate our side of the equation, but sometimes the problem lies in even deeper spaces, like app updates from an entirely different app.
Re: Simpson's paradox
#23I was reading the example of UC Berkely appearing to have gender bias in the admissions and read the following:
“it showed that women tended to apply to more competitive departments with lower rates of admission, even among qualified applicants (such as in the English department), whereas men tended to apply to less competitive departments with higher rates of admission (such as in the engineering department)”
That’s the opposite of what I would expect, I’d expect that English and the arts in general would be a lot easier to get into than stem, that’s how it is in Australia
Edit: When I say get into I mean get into university, not getting into the industry
Re: Simpson's paradox
#24That visualization was so effective I didn't need to read the wiki at all. I get it just from that.
Re: Simpson's paradox
#25I once encountered this in the real world as a data analyst a long time ago. I was working at an e-commerce company, called The Hut Group, and the whole year our marketing team had been saying our marketing cost of goods sold (the percentage of our revenue we needed to spend on marketing) had been declining across every product category. But at year end, the execs were shocked to realize that our cost of goods sold h…
It’s actually surprisingly common. You can even find it in “classical” toy datasets like Iris: https://github.com/DataForScience/Causality/blob/master/1.2%...
Covid vaccination rates and deaths were rather famously subject to it. E.g. some combination of stats like “most covid deaths were vaccinated individuals”, “vaccination reduces death rate”, and “population segment with lowest vaccination rates has lowest covid death rates.” were all true at the same time.
Re: Simpson's paradox
#26I once encountered this in the real world as a data analyst a long time ago. I was working at an e-commerce company, called The Hut Group, and the whole year our marketing team had been saying our marketing cost of goods sold (the percentage of our revenue we needed to spend on marketing) had been declining across every product category. But at year end, the execs were shocked to realize that our cost of goods sold h…
Pretty much every dataset I work with as an SRE is full of these paradoxes. One classic published example comes from Google: A network engineer took a trip to Indonesia or something (can't find the citation to confirm the exact tale), noticed the service was slow, and when asking around everyone said "that's how its always been." Basically the local cellular networks are slow and off island fiber connects are saturat…
But your example isn‘t a case of Simpson‘s Paradox (which is purely statistical), but Jevons Paradox (which is about human behaviour and economics).
Re: Simpson's paradox
#27I was reading the example of UC Berkely appearing to have gender bias in the admissions and read the following: “it showed that women tended to apply to more competitive departments with lower rates of admission, even among qualified applicants (such as in the English department), whereas men tended to apply to less competitive departments with higher rates of admission (such as in the engineering department)” That’s…
Competitiveness is one measure of difficulty, but there are others. Engineering departments tend to be qualitatively more difficult to get into than the humanities, which are quantitatively more difficult.
Re: Simpson's paradox
#28I once encountered this in the real world as a data analyst a long time ago. I was working at an e-commerce company, called The Hut Group, and the whole year our marketing team had been saying our marketing cost of goods sold (the percentage of our revenue we needed to spend on marketing) had been declining across every product category. But at year end, the execs were shocked to realize that our cost of goods sold h…
every time I hear about examples of simpson in peactice, I don't get what lesson to learn
marketting team overoptimized, so non-nutrition demand fell?
drop nutrition from line of products, so that you're both efficient in products you do and overall?
these metrics are insufficient and it's better to look at gross change rather than ratios?
I have no idea
Re: Simpson's paradox
#29I once encountered this in the real world as a data analyst a long time ago. I was working at an e-commerce company, called The Hut Group, and the whole year our marketing team had been saying our marketing cost of goods sold (the percentage of our revenue we needed to spend on marketing) had been declining across every product category. But at year end, the execs were shocked to realize that our cost of goods sold h…
every time I hear about examples of simpson in peactice, I don't get what lesson to learn marketting team overoptimized, so non-nutrition demand fell? drop nutrition from line of products, so that you're both efficient in products you do and overall? these metrics are insufficient and it's better to look at gross change rather than ratios? I have no idea
I think the last one is closest. I’d go with: “finance team should look at the gross change”, if that’s what matters for them.
Re: Simpson's paradox
#30Earlier quoted context omitted.
Pretty much every dataset I work with as an SRE is full of these paradoxes. One classic published example comes from Google: A network engineer took a trip to Indonesia or something (can't find the citation to confirm the exact tale), noticed the service was slow, and when asking around everyone said "that's how its always been." Basically the local cellular networks are slow and off island fiber connects are saturat…
But your example isn‘t a case of Simpson‘s Paradox (which is purely statistical), but Jevons Paradox (which is about human behaviour and economics).
Good point! I'm just a humble Linux sysadmin dubbed "SRE" who slept through Stats for Engineers and now pays the price every week dealing with SWE eager to blame me for their mistakes.