Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The math that explains why bell curves are everywhere

quantamagazine.org

Post reply on HN
Print view View original

51–60 of 133 posts

Re: The math that explains why bell curves are everywhere

#51

Post by thaumasiotes »

jibal wrote: post #48
tsunamifury wrote: post #25

Bell curves are everywhere because all distributions of any properties clump in some way at some level. The basics of any probability shows this. The result is you “seeing” bell curves everywhere. Aka clumps. This is a tautology to the extreme.

First, every mathematical theorem is a tautology ... don't conflate "tautological" with "obvious". Second, your "aka" is incorrect --- there is all sorts of clumping that is not a normal distribution.

As I'm sure tsunamifury would agree, it is incredibly common for people to label "bell curves" by eyeball, regardless of whether they are normal curves. To most people, "clumping" in a one-dimensional spectrum is all they mean by the phrase "bell curve".

Re: The math that explains why bell curves are everywhere

#52

Post by alanbernstein »

hilliardfarmer wrote: post #42
bicepjai wrote: post #40

This is one of my favorite philosophical questions to ponder. I always ask it in interviews as a warmup to get their thoughts. I’ve noticed that interviewees often curl up, thinking it’s a technical question, so I’ve been modifying the question one after the other to make it less scary. The interviews are for data scientist roles.

A lot of times I can't tell if I'm the idiot or if everyone else is. Says that this isn't an interesting question at all and the article was horrible. I studied data science for a few years but I'm no expert, but it seems pretty obvious to me that if you make a series of 50/50 choices randomly, that's the shape you end up with and there's really nothing more interesting about it than that.

I don't think "obvious" is the right word here. It makes perfect sense when you understand it, but it's not a conclusion that most people could come to immediately without detailed, assisted study.

Re: The math that explains why bell curves are everywhere

#53

Post by abetusk »

Buttons840 wrote: post #44
bicepjai wrote: post #40

This is one of my favorite philosophical questions to ponder. I always ask it in interviews as a warmup to get their thoughts. I’ve noticed that interviewees often curl up, thinking it’s a technical question, so I’ve been modifying the question one after the other to make it less scary. The interviews are for data scientist roles.

I haven't read the article, but my understanding is that a normal curve results from summing several samples from most common probability distributions, and also a normal curve results from summing many normal curves. All summation roads lead to normal curves. (There might be an exception for weird probability distributions that do not have a mean; I was surprised when I learned these exist.) Life is full of sums. He…

Say I have N independent and identically distributed random variables with finite mean. Assuming the sum converges to a distribution, what is the distribution they converge to?

Re: The math that explains why bell curves are everywhere

#54

Post by abetusk »

WCSTombs wrote: post #34

It's not a bad article, but I have to point something out: > Laplace distilled this structure into a simple formula, the one that would later be known as the central limit theorem. No matter how irregular a random process is, even if it’s impossible to model, the average of many outcomes has the distribution that it describes. “It’s really powerful, because it means we don’t need to actually care what is the distribu…

The fact the article said that is a gross error. You've identified the issue head on.

The sum of independent identically distributed random variables, if they converge at all, converge to a Levy stable distribution (aka fat-tailed, heavy tailed, power law). In this sense, Levy stable distributions are more "normal" than the normal distribution. They also show up with regular frequency all over nature.

As you point out, infinite variance might be dismissed but, in practice, this just ends up getting larger and larger "outliers" as one keeps drawing from the distribution. Infinities are, in effect, a "verb" and so an infinite variance, in this context, just means the distributions spits out larger and larger numbers the more you sample from it.

Re: The math that explains why bell curves are everywhere

#55

Post by tsunamifury »

thaumasiotes wrote: post #51
jibal wrote: post #48

Earlier quoted context omitted.

First, every mathematical theorem is a tautology ... don't conflate "tautological" with "obvious". Second, your "aka" is incorrect --- there is all sorts of clumping that is not a normal distribution.

As I'm sure tsunamifury would agree, it is incredibly common for people to label "bell curves" by eyeball, regardless of whether they are normal curves. To most people, "clumping" in a one-dimensional spectrum is all they mean by the phrase "bell curve".

[dead]

Re: The math that explains why bell curves are everywhere

#56

Post by D-Machine »

thaumasiotes wrote: post #51
jibal wrote: post #48

Earlier quoted context omitted.

First, every mathematical theorem is a tautology ... don't conflate "tautological" with "obvious". Second, your "aka" is incorrect --- there is all sorts of clumping that is not a normal distribution.

As I'm sure tsunamifury would agree, it is incredibly common for people to label "bell curves" by eyeball, regardless of whether they are normal curves. To most people, "clumping" in a one-dimensional spectrum is all they mean by the phrase "bell curve".

This was sort of my reading as well: I took "clumping" to mean "bump-shaped".

Re: The math that explains why bell curves are everywhere

#57

Post by Buttons840 »

abetusk wrote: post #53
Buttons840 wrote: post #44

Earlier quoted context omitted.

I haven't read the article, but my understanding is that a normal curve results from summing several samples from most common probability distributions, and also a normal curve results from summing many normal curves. All summation roads lead to normal curves. (There might be an exception for weird probability distributions that do not have a mean; I was surprised when I learned these exist.) Life is full of sums. He…

Say I have N independent and identically distributed random variables with finite mean. Assuming the sum converges to a distribution, what is the distribution they converge to?

A normal distribution.

Re: The math that explains why bell curves are everywhere

#58

Post by [deleted] »

parpfish wrote: post #4
mikrl wrote: post #3

Great article. Personally I have been learning more about the mathematics of beyond-CLT scenarios (fat tails, infinite variance etc) The great philosophical question is why CLT applies so universally. The article explains it well as a consequence of the averaging process. Alternatively, I’ve read that natural processes tend to exhibit Gaussian behaviour because there is a tendency towards equilibrium: forces, homeost…

As to ye philosophy of “why” the CLT gives you normals, my hunch is that it’s because there’s some connection between: a) the CLT requires samples drawn from a distribution with finite mean and variance and b) the Gaussian is the maximum entropy distribution for a particular mean and variance I’d be curious about what happens if you starting making assumptions about higher order moments in the distro

[deleted]

Re: The math that explains why bell curves are everywhere

#59

Post by abetusk »

Buttons840 wrote: post #57
abetusk wrote: post #53

Earlier quoted context omitted.

Say I have N independent and identically distributed random variables with finite mean. Assuming the sum converges to a distribution, what is the distribution they converge to?

A normal distribution.

Levy stable [0].

If I had made the extra condition that the random variables had finite variance, you'd be correct. Without the finite variance condition, the distribution is Levy stable.

Levy stable distributions can have finite mean but infinite variance. They can also have infinite mean and infinite variance. Only in the finite mean and finite variance case does it imply a Gaussian.

Levy stable distributions are also called "fat-tailed", "heavy-tailed" or "power law" distributions. In some sense, Levy stable distributions are more normal than the normal distribution. It might be tempting to dismiss the infinite variance condition but, practically, this just means you get larger and larger numbers as you draw from the distribution.

This was one of Mandelbrot's main positions, that power laws were much more common than previously thought and should be adopted much more readily.

As an aside, if you do ever get asked this in an interview, don't expect to get the job if you answer correctly.

[0] https://en.wikipedia.org/wiki/L%C3%A9vy_distribution

Re: The math that explains why bell curves are everywhere

#60

Post by abetusk »

fsckboy wrote: post #37
AndrewKemendo wrote: post #10

Earlier quoted context omitted.

That’s exactly the right take and the article proves it: Statisticians love averages so everywhere that could be sampled as a normal distribution will be presented as one The median is actually more descriptive and power law is equally as pervasive if not more

combining repeated samples of any distribution* (any population density fuction including power law distributions) will converge to the normal distribution, that's why it appears everywhere. * excluding bizarre degenerates like constants or impulse functions

No, that's not correct. Sums of power law distributions can converge to power low tailed distributions, not normal distributions.
Post reply on HN