Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The math that explains why bell curves are everywhere

quantamagazine.org

Post reply on HN
Print view View original

101–110 of 133 posts

Re: The math that explains why bell curves are everywhere

#101

Post by kurlberg »

srean wrote: post #80

The way I understand this is that adding of random variables is a smoothening operation on their densities (more generally the distributions, but let me speak of densities only). A little more formally, additions over random variables are convolutions of their densities. Repeated additions are repeated convolutions. A single convolution can be understood as a matrix multiplication by a specific symmetric matrix. Repe…

Convolution alone does not smooth. Eg consider a random variable supported on the pts 0 and 1 (delta masses at 2 pts.) No matter how many convolutions you do, you still have support on integers - not smooth at all. You need appropriate rescaling for a gaussian.

Also, convolving a distribution with itself is NOT a linear operation, hence cannot be described by a matrix multiplication with a fixed matrix.

Re: The math that explains why bell curves are everywhere

#102

Post by AndrewKemendo »

abetusk wrote: post #60
fsckboy wrote: post #37

Earlier quoted context omitted.

combining repeated samples of any distribution* (any population density fuction including power law distributions) will converge to the normal distribution, that's why it appears everywhere. * excluding bizarre degenerates like constants or impulse functions

No, that's not correct. Sums of power law distributions can converge to power low tailed distributions, not normal distributions.

No use arguing with them they don’t have enough mathematical understanding to understand what they’re saying

Re: The math that explains why bell curves are everywhere

#103

Post by stevenwoo »

Sharlin wrote: post #85
CoastalCoder wrote: post #81

Earlier quoted context omitted.

> Anyone familiar with linear algebra will know that repeated matrix multiplication by non degenerate matrices reveals it's eigenvectors. TIL that I'm not "familiar" with linear algebra ;) But seriously, thanks for sharing that knowledge.

Yeah, I don't think this was revealed on my undergrad linalg course, and neither during all my years of using linalg in computer graphics =D

I remember my professor talking about eigenvectors in Linear Algebra and it's been 50 years - though I barely remember anything else from that class. It was taught very early on in the course and eventually we used them all the time to solve problems.

Re: The math that explains why bell curves are everywhere

#104

Post by Sharlin »

stevenwoo wrote: post #103
Sharlin wrote: post #85

Earlier quoted context omitted.

Yeah, I don't think this was revealed on my undergrad linalg course, and neither during all my years of using linalg in computer graphics =D

I remember my professor talking about eigenvectors in Linear Algebra and it's been 50 years - though I barely remember anything else from that class. It was taught very early on in the course and eventually we used them all the time to solve problems.

Yes, I was taught about eigenvectors but not that they’re a fixpoint of matmul. At least I don’t think so.

Re: The math that explains why bell curves are everywhere

#105

Post by Sharlin »

srean wrote: post #89
Sharlin wrote: post #88

Earlier quoted context omitted.

Their point was that "familiarity" apparently means different things for different people :P Someone using linalg in computer graphics applications may say they're familiar with it even though they've never heard the term "eigenvector". I'm not actually sure about what you mean – how does repeated multiplication reveal eigenvectors?

Consider a diagonalizable matrix A. For example, a real symmetric matrix. Start with any vector b and keep multiplying it with A. A A A ... A b The vector that the result will converge to is a scaled version of one of the eigenvectors of the matrix A. But which one ? The one with the largest eigenvalue among all eigenvectors not orthogonal to b. https://en.wikipedia.org/wiki/Power_iteration

Ah… that "diagonalizable" is doing some heavy lifting there! I was wondering how exactly you’re going to make, say, a rotation matrix to converge anything to anything that’s not already an eigenvector. And rotation matrices certainly aren’t degenerate! Though apparently non-diagonalizable matrices can be called defective which is such a dismissive term :( Poor rotation matrices, why are they dissed so?!

Re: The math that explains why bell curves are everywhere

#106

Post by srean »

kurlberg wrote: post #101
srean wrote: post #80

The way I understand this is that adding of random variables is a smoothening operation on their densities (more generally the distributions, but let me speak of densities only). A little more formally, additions over random variables are convolutions of their densities. Repeated additions are repeated convolutions. A single convolution can be understood as a matrix multiplication by a specific symmetric matrix. Repe…

Convolution alone does not smooth. Eg consider a random variable supported on the pts 0 and 1 (delta masses at 2 pts.) No matter how many convolutions you do, you still have support on integers - not smooth at all. You need appropriate rescaling for a gaussian. Also, convolving a distribution with itself is NOT a linear operation, hence cannot be described by a matrix multiplication with a fixed matrix.

You are absolutely right. Even edge detection can be written as a convolution. That's why I mention averaging.

I address scaling, very peripherally, towards the end. Of course, depending on how you scale you end up with distinctly different limit laws.

Re: The math that explains why bell curves are everywhere

#107

Post by srean »

Sharlin wrote: post #105
srean wrote: post #89

Earlier quoted context omitted.

Consider a diagonalizable matrix A. For example, a real symmetric matrix. Start with any vector b and keep multiplying it with A. A A A ... A b The vector that the result will converge to is a scaled version of one of the eigenvectors of the matrix A. But which one ? The one with the largest eigenvalue among all eigenvectors not orthogonal to b. https://en.wikipedia.org/wiki/Power_iteration

Ah… that "diagonalizable" is doing some heavy lifting there! I was wondering how exactly you’re going to make, say, a rotation matrix to converge anything to anything that’s not already an eigenvector. And rotation matrices certainly aren’t degenerate! Though apparently non-diagonalizable matrices can be called defective which is such a dismissive term :( Poor rotation matrices, why are they dissed so?!

Love them, those rotation matrices.

Take logarithm of the eigenvalues and you get back the angle. This to me had solidified the notion that angles are essentially a logarithmic notion ... Made more rigorous by the notion of exponential maps

Re: The math that explains why bell curves are everywhere

#108

Post by bux93 »

The article doesn't share the actual math, but also not the relatively easy intuition. When you roll a pair of dice, there are more combinations that add up to 7 than any other number. Change the numbers on the dice (change the 1 to a 6, e.g.), there's again more combinations that add up to some numbers than to others. The histogram of the number of combinations that add up to different results is a bell curve. That's why it pops up everywhere you have addition of independent events. It's sad that even introductory statistics courses skip this simple intuition.

Re: The math that explains why bell curves are everywhere

#109

Post by AlexCornila »

bux93 wrote: post #108

The article doesn't share the actual math, but also not the relatively easy intuition. When you roll a pair of dice, there are more combinations that add up to 7 than any other number. Change the numbers on the dice (change the 1 to a 6, e.g.), there's again more combinations that add up to some numbers than to others. The histogram of the number of combinations that add up to different results is a bell curve. That'…

Yeah that is definitely not the relatively easy intuition for this. The relatively easy intuition comes from learning about the Bernoulli trials, binomial distribution and Pascal triangle. Once you understand those you understand why normal distribution is so prevalent. Or just watch this https://youtu.be/AwEaHCjgeXk?si=tV72uauquCHvzkNE

Re: The math that explains why bell curves are everywhere

#110

Post by dwpdwpdwpdwpdwp »

Francis Galton, namesake of the Galton Board,(https://en.wikipedia.org/wiki/Galton_board) had this to say of the Central Limit Theorem:

"Order in Apparent Chaos.-I know of scarcely any-, thing so apt to impress the imagination as the wonderful form of cosmic order expressed by the " Law of Frequency of Error." The law would have been personified by the Greeks and deified, if they had known of it. It reigns with serenity and • in complete self-effacement amidst the wildest confusion. The huger the mob, and the greater the apparent anarchy, the more perfect is its sway. It is the supreme law of Unreason."

https://galton.org/cgi-bin/searchImages/galton/search/books/...

Post reply on HN