Histograms for Probability Density Estimation: A Primer
vvanirudh.github.io
Histograms for Probability Density Estimation: A Primer
1–10 of 13 posts
Re: Histograms for Probability Density Estimation: A Primer
#2If the data is continuous, use kernel density estimation (KDE) instead of histograms to visualize the probability density, since KDE will give a smoother fit. A similar idea is to fit a mixture of normals -- there are numerous R packages for this and sklearn.mixture.GaussianMixture in SciPy.
Re: Histograms for Probability Density Estimation: A Primer
#3If the data is continuous, use kernel density estimation (KDE) instead of histograms to visualize the probability density, since KDE will give a smoother fit. A similar idea is to fit a mixture of normals -- there are numerous R packages for this and sklearn.mixture.GaussianMixture in SciPy.
Yep! The next post would be on Kernel density estimation -- wanted to start from histograms as they are still a useful tool in 1-D and 2-D density estimation, and you don't have to store the data either (unlike KDE)
Re: Histograms for Probability Density Estimation: A Primer
#4If the data is continuous, use kernel density estimation (KDE) instead of histograms to visualize the probability density, since KDE will give a smoother fit. A similar idea is to fit a mixture of normals -- there are numerous R packages for this and sklearn.mixture.GaussianMixture in SciPy.
Yep! The next post would be on Kernel density estimation -- wanted to start from histograms as they are still a useful tool in 1-D and 2-D density estimation, and you don't have to store the data either (unlike KDE)
I should have read to the end of your post:
'I will describe a very popular nonparametric method, Kernel Density Estimation, that also follows strategy 1 and is much more scalable to higher dimensions than histograms.'
Re: Histograms for Probability Density Estimation: A Primer
#5Earlier quoted context omitted.
Yep! The next post would be on Kernel density estimation -- wanted to start from histograms as they are still a useful tool in 1-D and 2-D density estimation, and you don't have to store the data either (unlike KDE)
I should have read to the end of your post: 'I will describe a very popular nonparametric method, Kernel Density Estimation, that also follows strategy 1 and is much more scalable to higher dimensions than histograms.'
Haha no worries!
Re: Histograms for Probability Density Estimation: A Primer
#6The best way to compute the empirical CDF (ECDF) is by sorting the data:
N = len(data)
X = sorted(data)
Y = np.arange(N)/N
plt.plot(X,Y)
Re: Histograms for Probability Density Estimation: A Primer
#7Why estimate PDF through histogram then convert to CDF, when one can estimate CDF directly? Doing so also avoids having to choose bin width that can have substantial impact.
Re: Histograms for Probability Density Estimation: A Primer
#8Why estimate PDF through histogram then convert to CDF, when one can estimate CDF directly? Doing so also avoids having to choose bin width that can have substantial impact.
Agreed -- very odd to use a parameter (bin width) in a nonparametric estimation. Just use the raw data. In numerical analysis, broadly speaking, integrals are stable while derivatives are wild; an empirical cdf is a nice smooth integral of the messy pdf.
Re: Histograms for Probability Density Estimation: A Primer
#9The best way to compute the empirical CDF (ECDF) is by sorting the data: N = len(data) X = sorted(data) Y = np.arange(N)/N plt.plot(X,Y) Technically, you should plot this with `plt.step`.
scipy even has a built-in method (scipy.stats.ecdf) for doing exactly this.
Re: Histograms for Probability Density Estimation: A Primer
#10The best way to compute the empirical CDF (ECDF) is by sorting the data: N = len(data) X = sorted(data) Y = np.arange(N)/N plt.plot(X,Y) Technically, you should plot this with `plt.step`.
scipy even has a built-in method (scipy.stats.ecdf) for doing exactly this.
Neat! That is so simple and in hindsight, makes a lot of sense. Thanks!