Understanding Convolutions in Probability
countbayesie.com
Understanding Convolutions in Probability
1–10 of 24 posts
Re: Understanding Convolutions in Probability
#2A somewhat related, visual guide to discrete 2d convolutions: https://ezyang.github.io/convolution-visualizer/
Re: Understanding Convolutions in Probability
#3A somewhat related, visual guide to discrete 2d convolutions: https://ezyang.github.io/convolution-visualizer/
Small correction: this is correlation operator. Deep learning wise they (correlation and convolution) are equivalent but in probability it will lead to incorrect results.
Re: Understanding Convolutions in Probability
#4"At a high level, in probability, a convolution is the way to determine the distribution of the sum of two random variables."
Probability of a certain sum value s is:
sum of probabilities of all (a,b) with a + b = s
(a: value from first input distribution. b: value from second input distribution)
with probability (a,b) = probability(a) * probability(b)
Re: Understanding Convolutions in Probability
#5A somewhat related, visual guide to discrete 2d convolutions: https://ezyang.github.io/convolution-visualizer/
Small correction: this is correlation operator. Deep learning wise they (correlation and convolution) are equivalent but in probability it will lead to incorrect results.
To elaborate, though the page does mention the distinction:
(Cross-)correlation is convolution with a reversed kernel (second signal) (or vice-versa, of course). (For discrete signals, reversing the kernel is just a swapping the indexes around; which makes absolutely no difference for deep-learning). Convolution is more "natural" because it's abelian, whereas swapping signal and kernel in cross-correlation time-reverses the result.
Re: Understanding Convolutions in Probability
#6I loved this article, it helped me understand: https://betterexplained.com/articles/intuitive-convolution/
Re: Understanding Convolutions in Probability
#7Re: Understanding Convolutions in Probability
#8A minor thought I found inspiring many years ago: the Central Limit Theorem may be considered a statement about dynamic processes in the functional space of distributions.
If you consider the operator of convolution (rescaling for unit variance), the normal distribution is the only attractor (under many "natural" choices of metrics).
Re: Understanding Convolutions in Probability
#9A minor thought I found inspiring many years ago: the Central Limit Theorem may be considered a statement about dynamic processes in the functional space of distributions. If you consider the operator of convolution (rescaling for unit variance), the normal distribution is the only attractor (under many "natural" choices of metrics).
It's actually not the only attractor -- it's just the only attractor once you restrict to finite variance. If you allow infinite variance, there can be others: https://en.wikipedia.org/wiki/Stable_distribution
Re: Understanding Convolutions in Probability
#10Mandatory link to a 3blue1brown video:
https://m.youtube.com/watch?v=KuXjwB4LzSA
For me, the way to understanding convolutions was to to put sample rand(0, 1) a bunch of times and bucket the samples, then plot the number of samples in a bucket. You get a more-or-less flat graph. Now if you add or multiply two variables, you get different shapes. Once you spent enough effort on trying to understand why, you derive convolutions.