Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The math that explains why bell curves are everywhere

quantamagazine.org

Post reply on HN
Print view View original

71–80 of 133 posts

Re: The math that explains why bell curves are everywhere

#71

Post by jibal »

thaumasiotes wrote: post #51
jibal wrote: post #48

Earlier quoted context omitted.

First, every mathematical theorem is a tautology ... don't conflate "tautological" with "obvious". Second, your "aka" is incorrect --- there is all sorts of clumping that is not a normal distribution.

As I'm sure tsunamifury would agree, it is incredibly common for people to label "bell curves" by eyeball, regardless of whether they are normal curves. To most people, "clumping" in a one-dimensional spectrum is all they mean by the phrase "bell curve".

This completely misses the point, which is that the central limit theorem says that it isn't just any old clumping, it's always the normal distribution. tsunamifury dismissed this strong finding as "tautology" because clumping is obvious ... but that it's always precisely a bell curve is far from obvious. Again,

> your "aka" is incorrect --- there is all sorts of clumping that is not a normal distribution.

That it's "incredibly common for people to label "bell curves" by eyeball, regardless of whether they are normal curves" is not just not relevant, it's anti-relevant ... the central limit theorem says that the distribution of the means is always a bell curve--a normal distribution--not merely a "bell curve".

Anyway, this is covered in far more detail in other comments and material elsewhere, so this is my last contribution.

Re: The math that explains why bell curves are everywhere

#72

Post by trhway »

mikrl wrote: post #3

Great article. Personally I have been learning more about the mathematics of beyond-CLT scenarios (fat tails, infinite variance etc) The great philosophical question is why CLT applies so universally. The article explains it well as a consequence of the averaging process. Alternatively, I’ve read that natural processes tend to exhibit Gaussian behaviour because there is a tendency towards equilibrium: forces, homeost…

>natural processes tend to exhibit Gaussian behaviour

to me it results of 2 factors - 1. Gaussian is the max entropy for a distribution with a given variance and 2. variance is the model of energy-limited behavior whereis physical processes are always under some energy limits. Basically it is the 2nd law.

Re: The math that explains why bell curves are everywhere

#73

Post by [deleted] »

btilly wrote: post #70
mikrl wrote: post #3

Great article. Personally I have been learning more about the mathematics of beyond-CLT scenarios (fat tails, infinite variance etc) The great philosophical question is why CLT applies so universally. The article explains it well as a consequence of the averaging process. Alternatively, I’ve read that natural processes tend to exhibit Gaussian behaviour because there is a tendency towards equilibrium: forces, homeost…

The key principle is that you get CLT when a bunch of random factors add. Which happens in lots of places. In finance, the effects of random factors tend to multiply. So you get a log-normal curve. As Taleb points out, though, the underlying assumptions behind log-normal break in large market movements. Because in large movements, things that were uncorrelated, become correlated. Resulting in fat tails, where extreme…

[deleted]

Re: The math that explains why bell curves are everywhere

#75

Post by srean »

A result of broader applicability is that of convergence to infinitely divisible distributions, more generally the stable distributions

https://en.wikipedia.org/wiki/Infinite_divisibility_(probabi...

https://en.wikipedia.org/wiki/Stable_distribution

This applies even when the variance is not finite.

Note independence and identical nature of distribution is not necessary for Central Limit Theorem to hold. It is a sufficient condition, not a necessary one, however, it does speed up the convergence a lot.

Gaussian distribution is a special case of the infinitely divisible distribution and is the most analytically tractable one in that family.

Whereas, averaging gives you Gaussian as long as the original distribution is somewhat benign, the MAX operator also has nice limiting properties. They converge to one of three forms of limiting distributions, Gumbel being one of them.

The general form of the limiting distributions when you take MAX of a sufficiently large sample are the extreme value distributions

https://en.wikipedia.org/wiki/Generalized_extreme_value_dist...

Very useful for studying record values -- severest floods, world records of 100m sprints, world records of maximum rainfall in a day etc

Re: The math that explains why bell curves are everywhere

#76

Post by srean »

btilly wrote: post #70
mikrl wrote: post #3

Great article. Personally I have been learning more about the mathematics of beyond-CLT scenarios (fat tails, infinite variance etc) The great philosophical question is why CLT applies so universally. The article explains it well as a consequence of the averaging process. Alternatively, I’ve read that natural processes tend to exhibit Gaussian behaviour because there is a tendency towards equilibrium: forces, homeost…

The key principle is that you get CLT when a bunch of random factors add. Which happens in lots of places. In finance, the effects of random factors tend to multiply. So you get a log-normal curve. As Taleb points out, though, the underlying assumptions behind log-normal break in large market movements. Because in large movements, things that were uncorrelated, become correlated. Resulting in fat tails, where extreme…

Some correlations are fine though, there are versions of CLT that applies even when there are benign correlations.

https://en.wikipedia.org/wiki/Central_limit_theorem#Dependen...

I know you know that and were just simplifying. Just wanted this fact to be better known for practitioners. Your comment on multiplicative processes is spot on.

I say more here

https://news.ycombinator.com/item?id=47437152

It's bit of a shame that these other limiting distributions are not as tractable as the Gaussian.

Re: The math that explains why bell curves are everywhere

#77

Post by Epa095 »

bandrami wrote: post #74

I flinch at "everywhere", particularly when people keep asserting they are places that they aren't (and in fact can't be). Nothing with a hard zero can be normally distributed, for instance, but people will keep insisting quantities with a hard zero are.

Is this not just a linguistic issue, where people say normal distributed but actually mean approximate or assumed normality? Its not like height is normally distributed (there is nobody 8 feet tall), but its not like the distribution bares no resemblance to the normal distribution either, and in a colloquial sense the term seems to he used more freely than the mathematical defined term.

Re: The math that explains why bell curves are everywhere

#78

Post by roenxi »

BobbyTables2 wrote: post #14
fritzo wrote: post #7

Hot take: bell curves are everywhere exactly because the math is simple. The causal chain is: the math is simple -> teachers teach simple things -> students learn what they're taught -> we see the world in terms of concepts we've learned. The central limit theorem generalizes beyond simple math to hard math: Levy alpha stable distributions when variance is not finite, the Fisher-Tippett-Gnedenko theorem and Gumbel/Fr…

It also took me a little while to realize “least squares” and MMSE approaches were not necessarily the “correct” way to do things but just “one thing we actually know how to do” because everything else is much harder. We can use Calculus to do so much but also so little…

That isn't the case; mathematicians will do pages of calculations (particularly and especially the statisticians) if they can prove one approach is technically superior to another. These people, as a class, are the crazies who invented matrix multiplication. Something like MMSE is used because it provably optimum properties for estimating a posterior distribution.

It is certainly possible that there are complex approaches that the statisticians have not discovered or don't teach because they are too complicated, but they had a big fight about which techniques were provably superior early in the discipline's history and the choices of what got standardised on weren't because of ease of calculation. It has actually been quite interesting how little interest the statisticians are likely to be taking in things like the machine learning revolution since the mathematics all seems pretty amenable to last century's techniques despite orders of magnitude differences in the data being handled.

Re: The math that explains why bell curves are everywhere

#79

Post by hodgehog11 »

bicepjai wrote: post #40

This is one of my favorite philosophical questions to ponder. I always ask it in interviews as a warmup to get their thoughts. I’ve noticed that interviewees often curl up, thinking it’s a technical question, so I’ve been modifying the question one after the other to make it less scary. The interviews are for data scientist roles.

It's amazing that you find so many that are uncomfortable with this question. I literally teach a first-year data science course and I ask the students this very question. I spend half a lecture on it and put it in their assessment.

This is one of the most fundamental things to understand in statistics. If you don't have at least some degree of comfort with this, you have no business working with data in a professional capacity.

Re: The math that explains why bell curves are everywhere

#80

Post by srean »

The way I understand this is that adding of random variables is a smoothening operation on their densities (more generally the distributions, but let me speak of densities only).

A little more formally, additions over random variables are convolutions of their densities. Repeated additions are repeated convolutions.

A single convolution can be understood as a matrix multiplication by a specific symmetric matrix. Repeated convolutions are therefore repeated matrix multiplications.

Anyone familiar with linear algebra will know that repeated matrix multiplication by a non degenerate matrix reveals it's eigenvectors.

The Gaussian distribution is such an eigenvector. Just like an eigenvector, it is also a fixed point -- multiplying again by the same matrix wil lead to the same vector, just scaled. The Gaussian distribution convolved is again a Gaussian distribution.

The addition operation in averaging is a matrix multiplication in the distribution space and the division by the the 'total' in the averaging takes care of the scaling.

Linear algebra is amazing.

Pagerank is an eigenvector of the normalised web adjacency matrix. Gaussian distribution is the eigenvector of the infinite averaging matrix. Essentially the same idea.

Post reply on HN