Live data from Hacker News

The math that explains why bell curves are everywhere

quantamagazine.org

11–20 of 133 posts

Re: The math that explains why bell curves are everywhere

#11

I hate Quanta a lot a vast amount of fluff for less than a college statistics professor would (hopefully) be able to impart with a chalkboard in 10 minutes, when Quanta has the ability to prepare animated diagrams like 3Blue1Brown but chooses not to use it they could go down myriad paths, like how it provides that random walks on square lattices are asymptotically isotropic, or give any other simple easy-to-understan…

[deleted]

Re: The math that explains why bell curves are everywhere

#12
post #4
post #3

Great article. Personally I have been learning more about the mathematics of beyond-CLT scenarios (fat tails, infinite variance etc) The great philosophical question is why CLT applies so universally. The article explains it well as a consequence of the averaging process. Alternatively, I’ve read that natural processes tend to exhibit Gaussian behaviour because there is a tendency towards equilibrium: forces, homeost…

As to ye philosophy of “why” the CLT gives you normals, my hunch is that it’s because there’s some connection between: a) the CLT requires samples drawn from a distribution with finite mean and variance and b) the Gaussian is the maximum entropy distribution for a particular mean and variance I’d be curious about what happens if you starting making assumptions about higher order moments in the distro

IIRC the third moment defines a maxent distribution under certain conditions and with a fourth moment it becomes undefined? It's been awhile though.

If I'm remembering it correctly it's interesting to think about the ramifications of that for the moments.

Re: The math that explains why bell curves are everywhere

#14
post #7

Hot take: bell curves are everywhere exactly because the math is simple. The causal chain is: the math is simple -> teachers teach simple things -> students learn what they're taught -> we see the world in terms of concepts we've learned. The central limit theorem generalizes beyond simple math to hard math: Levy alpha stable distributions when variance is not finite, the Fisher-Tippett-Gnedenko theorem and Gumbel/Fr…

It also took me a little while to realize “least squares” and MMSE approaches were not necessarily the “correct” way to do things but just “one thing we actually know how to do” because everything else is much harder.

We can use Calculus to do so much but also so little…

Re: The math that explains why bell curves are everywhere

#15
post #4
post #3

Great article. Personally I have been learning more about the mathematics of beyond-CLT scenarios (fat tails, infinite variance etc) The great philosophical question is why CLT applies so universally. The article explains it well as a consequence of the averaging process. Alternatively, I’ve read that natural processes tend to exhibit Gaussian behaviour because there is a tendency towards equilibrium: forces, homeost…

As to ye philosophy of “why” the CLT gives you normals, my hunch is that it’s because there’s some connection between: a) the CLT requires samples drawn from a distribution with finite mean and variance and b) the Gaussian is the maximum entropy distribution for a particular mean and variance I’d be curious about what happens if you starting making assumptions about higher order moments in the distro

The standard framing defines the Gaussian as this special object with a nice PDF, then presents the CLT as a surprising property it happens to have. But convolution of densities is the fundamental operation. If you keep convolving any finite-variance distribution with itself, the shape converges, and we called the limit "normal." The Gaussian is a fixed point of iterated convolution under √n rescaling. It earned its name by being the thing you inevitably get, not by having elegant closed-form properties.

The most interesting assumptions to relax are the independence assumptions. They're way more permissive than the textbook version suggests. You need dependence to decay fast enough, and mixing conditions (α-mixing, strong mixing) give you exactly that: correlations that die off let the CLT go through essentially unchanged. Where it genuinely breaks is long-range dependence -fractionally integrated processes, Hurst parameter above 0.5, where autocorrelations decay hyperbolically instead of exponentially. There the √n normalization is wrong, you get different scaling exponents, and sometimes non-Gaussian limits.

There are also interesting higher order terms. The √n is specifically the rate that zeroes out the higher-order cumulants. Skewness (third cumulant) decays at 1/√n, excess kurtosis at 1/n, and so on up. Edgeworth expansions formalize this as an asymptotic series in powers of 1/√n with cumulant-dependent coefficients. So the Gaussian is the leading term of that expansion, and Edgeworth tells you the rate and structure of convergence to it.

Re: The math that explains why bell curves are everywhere

#16
post #7

Hot take: bell curves are everywhere exactly because the math is simple. The causal chain is: the math is simple -> teachers teach simple things -> students learn what they're taught -> we see the world in terms of concepts we've learned. The central limit theorem generalizes beyond simple math to hard math: Levy alpha stable distributions when variance is not finite, the Fisher-Tippett-Gnedenko theorem and Gumbel/Fr…

The CLT is everywhere because convolution/adding independentish random variables is a super common thing to do.

Re: The math that explains why bell curves are everywhere

#17
post #6

I hate Quanta a lot a vast amount of fluff for less than a college statistics professor would (hopefully) be able to impart with a chalkboard in 10 minutes, when Quanta has the ability to prepare animated diagrams like 3Blue1Brown but chooses not to use it they could go down myriad paths, like how it provides that random walks on square lattices are asymptotically isotropic, or give any other simple easy-to-understan…

A lot of times on HN when a math topic comes up that isn't about 3b1b, someone will jump in to say "this isn't as good as 3b1b". Last time I saw that, I was moved to comment: https://news.ycombinator.com/item?id=45800657 3b1b doesn't have the same goal as Quanta, or as introductory guides. It's actually not that great a teaching tool (it's truly great at what it is for, which is (a) appreciation and motivation, and (…

there is no getting around that learning math requires actually having to buckle down and read and do math . A video will not suffice.

Re: The math that explains why bell curves are everywhere

#18
post #6

Earlier quoted context omitted.

A lot of times on HN when a math topic comes up that isn't about 3b1b, someone will jump in to say "this isn't as good as 3b1b". Last time I saw that, I was moved to comment: https://news.ycombinator.com/item?id=45800657 3b1b doesn't have the same goal as Quanta, or as introductory guides. It's actually not that great a teaching tool (it's truly great at what it is for, which is (a) appreciation and motivation, and (…

there is no getting around that learning math requires actually having to buckle down and read and do math . A video will not suffice.

Couldn't agree more, which is why I think it's odd to suggest that a pop-sci magazine article is somehow a disservice that 3b1b would correct.

Re: The math that explains why bell curves are everywhere

#19
post #4
post #3

Great article. Personally I have been learning more about the mathematics of beyond-CLT scenarios (fat tails, infinite variance etc) The great philosophical question is why CLT applies so universally. The article explains it well as a consequence of the averaging process. Alternatively, I’ve read that natural processes tend to exhibit Gaussian behaviour because there is a tendency towards equilibrium: forces, homeost…

As to ye philosophy of “why” the CLT gives you normals, my hunch is that it’s because there’s some connection between: a) the CLT requires samples drawn from a distribution with finite mean and variance and b) the Gaussian is the maximum entropy distribution for a particular mean and variance I’d be curious about what happens if you starting making assumptions about higher order moments in the distro

It is the not knowing, the unknown unknowns and known unknowns which result in the max entropy distribution's appearance. When we know more, it is not Gaussian. That is known.

Re: The math that explains why bell curves are everywhere

#20

I hate Quanta a lot a vast amount of fluff for less than a college statistics professor would (hopefully) be able to impart with a chalkboard in 10 minutes, when Quanta has the ability to prepare animated diagrams like 3Blue1Brown but chooses not to use it they could go down myriad paths, like how it provides that random walks on square lattices are asymptotically isotropic, or give any other simple easy-to-understan…

3Blue1Brown

Seems a bit like Ted Talks. Lightweight popcorn for the simple minded.

Post reply on HN