Why you should be wary of relying on a single histogram of a data set
stats.stackexchange.com
Why you should be wary of relying on a single histogram of a data set
1–10 of 21 posts
Re: Why you should be wary of relying on a single histogram of a data set
#2Reminds me of http://en.wikipedia.org/wiki/Simpsons_paradox
Re: Why you should be wary of relying on a single histogram of a data set
#3As mentioned, one should really be using a kernel density plot instead of a histogram, except when there are already classes in the data.
In R, one can simply do:
library("ggplot2")
library("datasets")
ggplot(faithful, aes(x=eruptions)) + geom_density() + geom_rug()
ggplot(faithful, aes(x=eruptions)) + geom_histogram(binwidth=1)
Edit: Other plots mentioned in this discussion:
ggplot(faithful, aes(x = eruptions)) + stat_ecdf(geom = "step")
qqnorm (faithful$eruptions)
Re: Why you should be wary of relying on a single histogram of a data set
#4Interesting paradox. I haven't seen that many statisticians using just a histogram when determining whether a certain distribution fits data reasonably. Kernel Density Estimators are a much better choice (for continuous data, like the data in the post), but they are also affected by your choice of bandwidth. When it comes down to it, like going to the doctor, sometimes the best choice is to get a second (or third!) opinion. For what it's worth, drawing a QQ Plot (something I've seen in every statistical consultation I've ever done) reveals the dependent structure of the data immediately and obviously in the form of a perfect linear relationship between any two variables.
Re: Why you should be wary of relying on a single histogram of a data set
#5Yes, probability density estimation might be fun, but the simplest thing to do when comparing distributions, if you're worried about binning issues, is to plot their empirical cumulative distribution functions.
Re: Why you should be wary of relying on a single histogram of a data set
#6Interesting paradox. I haven't seen that many statisticians using just a histogram when determining whether a certain distribution fits data reasonably. Kernel Density Estimators are a much better choice (for continuous data, like the data in the post), but they are also affected by your choice of bandwidth. When it comes down to it, like going to the doctor, sometimes the best choice is to get a second (or third!) o…
Indeed, although Q-Q plots are very unlikely to be understood by people who don't have a good grasp of statistics, whereas a misleading histogram will be (and probably without knowledge of the caveats behind histograms).
Re: Why you should be wary of relying on a single histogram of a data set
#7As mentioned, one should really be using a kernel density plot instead of a histogram, except when there are already classes in the data. In R, one can simply do: library("ggplot2") library("datasets") ggplot(faithful, aes(x=eruptions)) + geom_density() + geom_rug() which gives a chart like this ( http://jean-francois.im/temp/eruptions-kde.png ). Contrast with: ggplot(faithful, aes(x=eruptions)) + geom_histogram(binw…
But then you would have to choose a certain kernel and assume the data conforms to that distribution which isn't always true.
Re: Why you should be wary of relying on a single histogram of a data set
#8As mentioned, one should really be using a kernel density plot instead of a histogram, except when there are already classes in the data. In R, one can simply do: library("ggplot2") library("datasets") ggplot(faithful, aes(x=eruptions)) + geom_density() + geom_rug() which gives a chart like this ( http://jean-francois.im/temp/eruptions-kde.png ). Contrast with: ggplot(faithful, aes(x=eruptions)) + geom_histogram(binw…
But then you would have to choose a certain kernel and assume the data conforms to that distribution which isn't always true.
Indeed, but that estimate is likely to be less misleading in most cases than a histogram(which is just a uniform kernel that is always aligned with bin boundaries).
Re: Why you should be wary of relying on a single histogram of a data set
#9This is what you should be doing:
plot(density(Annie), col="red")
lines(density(Brian), col="blue")
lines(density(Chris), col="green")
lines(density(Zoe), col="cyan")
Re: Why you should be wary of relying on a single histogram of a data set
#10Reminds me of http://en.wikipedia.org/wiki/Simpsons_paradox
Hmm, reminds me much more of Anscombe's quartet: http://en.wikipedia.org/wiki/Anscombe%27s_quartet