Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Probability-generating functions

entropicthoughts.com

Post reply on HN
Print view View original

41–50 of 54 posts

Re: Probability-generating functions

#41

Post by ysofunny »

beagle3 wrote: post #30
ysofunny wrote: post #14

Earlier quoted context omitted.

but isn't a characteristic function just "the" way to bridge the gap between sets, functions, and logic(? ...a 3way bridge!?) I mean, it was useful for me to think about like a translation between sets and logic (this variable x is in the set xor not) into functions (a function f(x) that returns 1 or true whenever x is in set S) how the heck is that a fourier transform!??

“Characterstic function” is (was) an overloaded term. What you described is more often referred to as an “indicator function” these days, with “characteristic functions” denoting the transform (Fourier, laplace, z - depending on context). Closely related to “moment generating functions” to the point of being almost interchangeable.

so the same thing but, characterisic function as I knew them before these posts is a rudimentary 2-variable finite version. point and line (but the line is a curve, a circle because e).

but the new and improved 21st century characteristic functions are n-variable and have a full continious spectrum of variables between zero (false) and one (true) but only potentially lest infinite realizes itself (which would make the theories illogical).

this way of thinking about this makes sense to me, even if it's ever so slighly wrong by some nitpickable point https://en.wikipedia.org/wiki/Moment-generating_function

Re: Probability-generating functions

#42

Post by schmidtleonard »

creata wrote: post #39
jamessb wrote: post #24

Earlier quoted context omitted.

Yes, this provides good intuition about why it is useful: the PDF of the sum of two random variables is the convolution of the original PDFs. A convolution is awkward to work with, but by the convolution theorem it is a multiplication in the Fourier domain. This immediately suggests that the Fourier transform of a PDF would be a useful thing to work with. If you don't say that this is what you are doing then it all s…

> the PDF of the sum of two random variables is the convolution of the original PDFs (Probably obvious to everyone reading, but the variables should be independent.)

But I'd rather assume the variables are independent and then blame statistics when I get the wrong answer!

Re: Probability-generating functions

#43

Post by mturmon »

In a footnote OP says:

> I’m not yet good enough to intuitively get why the curvature of the probability-generating would be related to variance, but I’d be happy to receive pointers here.

Here’s my intuition for this.

The characteristic function is the Fourier transform of the density.

If the density is in “t” units, the ch.f. is in f= 1/t units. It is the “inverse domain.” (I’m using “f” to suggest frequency, ie the Fourier coordinate.)

Of course it is not a simple coordinate transformation! But some intuition does carry over.

This is reflected in all sorts of ways. It’s one reason why the IFT formula is so functionally close to the FT formula.

Anyway.

Because of this, the behavior of the FT (ch.f.) very close to the origin (“f=0”) tells about the tails of the distribution (t = 1/f is large).

In particular, high curvature around the origin tells you the tails are heavy. That’s the variance.

This extends to the fourth moment. You can get even sharper curvature around the origin of the FT (ch.f. at f=0) with a large coefficient on the fourth order term. This corresponds to a large fourth moment of the pdf, or a high kurtosis.

It’s useful to recall that, because of analytic continuation, knowing all the derivatives at the one point f=0 determines the ch.f. everywhere, and thereby determines the complete density. This corresponds to the fact that knowing all the moments determines the full density.

So in a very real sense, you only need the ch.f. in a tight neighborhood of the origin!

(Provided all moments are finite.)

Re: Probability-generating functions

#44

Post by lr1970 »

fermisea wrote: post #28
KvanteKat wrote: post #9

For those interested in looking slightly more into the characteristic function, it may be worth pointing out that the characteristic function is equal to the Fourier-transform (with the sign of the argument being reversed) of the probability distribution in question. In my own experience teaching teaching probability theory to physicists and engineers, establishing this connection is often a good way of helping peopl…

As a physicist, the moment when everything just clicked was when I realised that connected Feynman diagrams were basically the cumulants of that distribution. Then almost everything in physics is about "what is the characteristic/moment/cumulant generating function?" and associated Legendre transforms

> As a physicist, the moment when everything just clicked was when I realised that connected Feynman diagrams were basically the cumulants of that distribution.

And the generating function of the cumulants is the logarithm of the generating function of the distribution (Fourier transform).

Re: Probability-generating functions

#45

Post by GemesAS »

fermisea wrote: post #28
KvanteKat wrote: post #9

For those interested in looking slightly more into the characteristic function, it may be worth pointing out that the characteristic function is equal to the Fourier-transform (with the sign of the argument being reversed) of the probability distribution in question. In my own experience teaching teaching probability theory to physicists and engineers, establishing this connection is often a good way of helping peopl…

As a physicist, the moment when everything just clicked was when I realised that connected Feynman diagrams were basically the cumulants of that distribution. Then almost everything in physics is about "what is the characteristic/moment/cumulant generating function?" and associated Legendre transforms

A little known bit of history is Feynman developed a diagrammatic method for expressing the moments of PGFs in his study of the stochastic theory of fission chains. This was before his work on QED. See:

https://www.osti.gov/biblio/1775045

Re: Probability-generating functions

#47

Post by credit_guy »

fermisea wrote: post #28
KvanteKat wrote: post #9

For those interested in looking slightly more into the characteristic function, it may be worth pointing out that the characteristic function is equal to the Fourier-transform (with the sign of the argument being reversed) of the probability distribution in question. In my own experience teaching teaching probability theory to physicists and engineers, establishing this connection is often a good way of helping peopl…

As a physicist, the moment when everything just clicked was when I realised that connected Feynman diagrams were basically the cumulants of that distribution. Then almost everything in physics is about "what is the characteristic/moment/cumulant generating function?" and associated Legendre transforms

Wow. I am not a physicist, but I use pdfs and moments and cumulants all the time. I came up with my own method to calculate cumulants for affine processes using some recursions, and they work. But if I hear you right, I might have stumbled upon something that Feynman did 70 years ago, and he probably did it better. Any good links you can recommend?

Re: Probability-generating functions

#48

Post by Sharlin »

sampo wrote: post #46
Sharlin wrote: post #33

Probably worth noting that as we know know, polynomials (over a field) are a vector space, not just convertible to one. The set of formal variables { x^0, x^1, x^2, … } is an orthogonal basis.

A basis yes, but not orthogonal.

How so? It’s impossible to express x^n as any linear combination of x^k, k != n.

Re: Probability-generating functions

#49

Post by sampo »

Sharlin wrote: post #48
sampo wrote: post #46

Earlier quoted context omitted.

A basis yes, but not orthogonal.

How so? It’s impossible to express x^n as any linear combination of x^k, k != n.

You have mixed up these two concepts:

https://en.wikipedia.org/wiki/Linear_independence

https://en.wikipedia.org/wiki/Orthogonal_basis

Post reply on HN