Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simulating and Visualising the Central Limit Theorem

blog.foletta.net

Post reply on HN
Print view View original

41–50 of 67 posts

Re: Simulating and Visualising the Central Limit Theorem

#41

Post by kqr »

jethkl wrote: post #28

There is an analogue of the CLT for extreme values. The Fisher–Tippett–Gnedenko theorem is the extreme-values analogue of the CLT: if the properly normalized maximum of an i.i.d. sample converges, it must be Gumbel, Fréchet, or Weibull—unified as the Generalized Extreme Value distribution. Unlike the CLT, whose assumptions (in my experience) rarely hold in practice, this result is extremely general and underpins meth…

There's also a more conservative rule similar to the CLT that works off of the definition of variance, and thus rests on no assumptions other than the existence of variance. Chebyshev's inequality tells us that the probability that any sample is more than k standard deviations away is bounded by 1/k².

In other words, it is possible (given sufficiently weird distributions) that not a single sample lands inside one standard deviation, but 75% of them must be inside two standard deviations, 88% inside three standard deviations, and so on.

There's also a one-sided version of it (Cantelli's inequality) which bounds the probability of any sample by 1/(1+k)², meaning at least 75 % of samples must be less than one standard deviation, 88% less than two standard deviations, etc.

Think of this during the next financial crisis when bank people no doubt will say they encountered "six sigma daily movements which should happen only once every hundred million years!!" or whatever. According to the CLT, sure, but for sufficiently odd distributions the Cantelli bound might be a more useful guide, and it says six sigma daily movements could happen as often as every fifty days.

Re: Simulating and Visualising the Central Limit Theorem

#42

Post by kqr »

firesteelrain wrote: post #34

“ You’re also likely not going to have the resources to take twenty-thousand different samples.” There are methods to calculate how many estimated samples you need. It’s not in the 20k unless your population is extremely high

> It’s not in the 20k unless your population is extremely high

Common misconception. Population size has almost nothing to do with the necessary sample size. (It does enter into the finite population correction factor, but that's only really relevant if you have a small population, not a large one.)

...actually, come to think of it, you meant to write "unless your population variance is extremely high", right?

Re: Simulating and Visualising the Central Limit Theorem

#44

Post by firesteelrain »

kqr wrote: post #42
firesteelrain wrote: post #34

“ You’re also likely not going to have the resources to take twenty-thousand different samples.” There are methods to calculate how many estimated samples you need. It’s not in the 20k unless your population is extremely high

> It’s not in the 20k unless your population is extremely high Common misconception. Population size has almost nothing to do with the necessary sample size. (It does enter into the finite population correction factor, but that's only really relevant if you have a small population, not a large one.) ...actually, come to think of it, you meant to write "unless your population variance is extremely high", right?

Right and the methods I had in mind were the usual ones: proportion-based, mean-based, and power analysis, depending on what’s being measured. Thanks for catching that

Re: Simulating and Visualising the Central Limit Theorem

#45

Post by selimthegrim »

nextos wrote: post #33
niemandhier wrote: post #2

Highly entertaining, here a little fun fact: there exist a generalisation of the central limit theorem for distributions without find out variance. For some reasons this is much less known, also the implications are vast. Via the detour of stable distributions and limiting distributions, this generalised central limit theorem plays an important role in the rise of power laws in physics.

Yes, came here to say the same thing. Telling people that the CLT makes strong assumptions is important. Otherwise, they might end up underestimating rare events, with potentially catastrophic consequences. There are also CLTs for product and max operators, aside from the sum. The Fundamentals of Heavy Tails: Properties, Emergence, and Estimation discusses these topics in a rigorous way, but without excessive mathema…

I was at the NeurIPS workshop and saw one of the talks towards the end and it was pretty good

Re: Simulating and Visualising the Central Limit Theorem

#46

Post by ivansavz »

I love the simulations. They are such a good way to learn STATS... you can still look at the theorem using math notation after, but if you've seen it work first using simulated random samples, then the math will make a lot more sense.

Here is a notebook with some more graphs and visualizations of the CLT: https://nobsstats.com/site/notebooks/28_random_samples/#samp...

runnable link: https://mybinder.org/v2/gh/minireference/noBSstats/main?labp...

Re: Simulating and Visualising the Central Limit Theorem

#47

Post by gus_massa »

> It’s very subjective, but I think the uniform stsrts looking reasonably good at a sample size of 8. The exponential however takes much longer to converge to a normal.

That's a good observation. The main idea behind the Central Limit Theorem is to take the Fourier Transform, operate and then go back. After that, after normalization the result is that the new distribution for the sum of N variables is something like

  Normal(X) + 1/N * "Skewness" * Something(X) + 1/N^2 * IDont * Remember(X) + ...
Where "Skewness" is a number defined in https://en.wikipedia.org/wiki/Skewness

The uniform distribution is symmetric, so skewness=0 and the correction decrease like 1/N^2.

The exponential distribution is very asymmetrical and and skewness!=0, so the main correction is like 1/N and takes longer to dissapear.

Re: Simulating and Visualising the Central Limit Theorem

#49

Post by ngriffiths »

lottin wrote: post #5

Looking at the R code in this article, I'm having a hard time understanding the appeal of tidyverse.

For me the appeal is less that tidyverse is great and more that the R standard library is horrible. It's full of esoteric names, inconsistent use and order of parameters, unreasonable default behavior, behavior that surprises you coming from other programming experience. It's all in a couple massive packages instead of broken up into manageable pieces.

Tidyverse is imperfect and it feels heavy-handed and awkward to replace all the major standard library functions, but Tidyverse stuff is way more ergonomic.

Re: Simulating and Visualising the Central Limit Theorem

#50

Post by antognini »

There's an interesting extension of the Central Limit Theorem called the Edgeworth Series. If you have a large but finite sample, the resulting distribution will be approximately Gaussian, but will deviate from a Gaussian distribution in a predictable way described by Hermite polynomials.

https://en.wikipedia.org/wiki/Edgeworth_series

Post reply on HN