Gaussian distributions form a monoid
izbicki.me
Gaussian distributions form a monoid
1–10 of 15 posts
Re: Gaussian distributions form a monoid
#2> "Gaussians are ubiquitous in learning algorithms because they accurately describe most data."
Amazing!
Uh, might want to reconsider and check that.
Re: Gaussian distributions form a monoid
#3> "Gaussians are ubiquitous in learning algorithms because they accurately describe most data." Amazing! Uh, might want to reconsider and check that.
This is especially true if you have multimodal distributions (such as the income datasets in the article). Although its true that there are simple algebraic properties that allow you to calculate the mean and variance of the total population given the same for sub-populations, it often isn't the case that this will be a good fit to the data. That being said, this is a useful property for parallelizing gaussian fits.
Re: Gaussian distributions form a monoid
#4For anyone else who didn't understand what the author meant by a binary operation on Gaussian distributions: he is really talking about combining sufficient statistics. In fact, this property is true for any exponential family (Gamma, Poisson, etc).
Re: Gaussian distributions form a monoid
#5> "Gaussians are ubiquitous in learning algorithms because they accurately describe most data." Amazing! Uh, might want to reconsider and check that.
Yeah NNT will call out the pitchforks for this guy. In fairness, though, in this area, this usually "works" (using the ML meaning of that verb).
Re: Gaussian distributions form a monoid
#6> "Gaussians are ubiquitous in learning algorithms because they accurately describe most data." Amazing! Uh, might want to reconsider and check that.
This is especially true if you have multimodal distributions (such as the income datasets in the article). Although its true that there are simple algebraic properties that allow you to calculate the mean and variance of the total population given the same for sub-populations, it often isn't the case that this will be a good fit to the data. That being said, this is a useful property for parallelizing gaussian fits.
> That being said, this is a useful property for parallelizing gaussian fits.
Might you be able to clarify this sentence? I have zero idea what might be meant:
> this is a useful property
What is the antecedent of "this", that is, in this phrases, what does "this" refer to?
> gaussian fits
What is a gaussian fit? I have no idea. I'm comfortable with the Lindeberg-Feller version of the central limit theorem, the weak and strong laws of large numbers, martingale theory, the martingale proof of the strong law of large numbers, the Radon-Nikodym theorem, and the fact that sample mean and variance are sufficient statistics for the Gaussian distribution, but, still, I can't even guess what a gaussian fit is.
> parallelizing
I can guess that what is meant by "parallelizing" is the computer software approach of having one program try to get some work done faster by starts several threads or tasks in one or several processor cores, processors, or computers. Okay. But what is it about "gaussian fits" that might commonly call for "parallelizing"?
Re: Gaussian distributions form a monoid
#7> "Gaussians are ubiquitous in learning algorithms because they accurately describe most data." Amazing! Uh, might want to reconsider and check that.
Yeah NNT will call out the pitchforks for this guy. In fairness, though, in this area, this usually "works" (using the ML meaning of that verb).
Maybe we just need to note
that mean and variance can
be useful in
using some data even if
the distribution of
that data is not Gaussian.
Re: Gaussian distributions form a monoid
#8Earlier quoted context omitted.
This is especially true if you have multimodal distributions (such as the income datasets in the article). Although its true that there are simple algebraic properties that allow you to calculate the mean and variance of the total population given the same for sub-populations, it often isn't the case that this will be a good fit to the data. That being said, this is a useful property for parallelizing gaussian fits.
> That being said, this is a useful property for parallelizing gaussian fits. Might you be able to clarify this sentence? I have zero idea what might be meant: > this is a useful property What is the antecedent of "this", that is, in this phrases, what does "this" refer to? > gaussian fits What is a gaussian fit ? I have no idea. I'm comfortable with the Lindeberg-Feller version of the central limit theorem, the weak…
Fitting a distribution to data is pretty common parlance in my experience, and there is even a wikipedia article with a relevant name [0].
I presume that the parallelisation point was with reference to the point made by the article, that the calculation of means and variances can be parallelised, so large datasets can be dealt with efficiently.
Is there something else you are missing?
Re: Gaussian distributions form a monoid
#9> "Gaussians are ubiquitous in learning algorithms because they accurately describe most data." Amazing! Uh, might want to reconsider and check that.
It's not necessary for the distributions to be normal for this approach to be relevant. A lot of the (very powerful) Gaussian machinery for e.g. graphical models, can still be rigorously applied to a many non-normal distributions by using the Nonparanormal transform:
http://jmlr.csail.mit.edu/papers/volume10/liu09a/liu09a.pdf
Re: Gaussian distributions form a monoid
#10Back when I was a grad student in physics we used similar thinking to prove the central limit theorem w/ the renormalization group.