Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I've stopped using box plots (2021)

nightingaledvs.com

Post reply on HN
Print view View original

231–240 of 258 posts

Re: I've stopped using box plots (2021)

#231

Post by jncfhnb »

sigmoid10 wrote: post #218
fnordpiglet wrote: post #170

Earlier quoted context omitted.

Sorry I don’t understand. The central limit theorem describe the distribution of the sample means from a population. It describes the distribution of the mean, not the distribution of the population itself. The shape of the distribution of the sample mean isn’t super interesting when you’re interested in the distribution of the samples themselves as a proxy for a population. So I’m not sure I understand your assertio…

I't appears you don't understand the central limit theorem fully. You gave the definition you find in textbooks, but you don't see how it applies to real world measurements and already explains your question. I can only recommend to visit a university level statistics course at this point. Maybe you will understand when you actually deal with some real data. Then you will indeed see its consequences pop up everywhere…

The GP here did not claim a poisson was the same thing as a Gaussian. They also don’t look similar.

As far as I can tell you’re making the introductory student error of thinking the central limit theorem means any sufficiently large sample makes a distribution look normal.

Re: I've stopped using box plots (2021)

#232

Post by seanhunter »

commathingy wrote: post #126
thaumasiotes wrote: post #105

Earlier quoted context omitted.

Is it? How? You could include a lot of little bells far from the single mode, but that's reading a little too much into the literal meaning of "single mode" - a "bimodal" distribution isn't one where the two most common values are both modes. It's one where there are two distinct local maxima. The tails to the left and right must asymptotically approach zero (or you don't have a smooth distribution, because you have…

The exponential distribution (modal value 0) is not bell shaped. If you don't like it's range of non-negative, then take some smooth mollification

Yes or the chi-square distribution with k=1 or 2 or any other of the gamma distributions[1] with the right parameters will have a shape that is one-sided with the mode at the lower extreme and no "low tail" in the normal sense.

[1] https://en.wikipedia.org/wiki/Gamma_distribution or https://stats.libretexts.org/Bookshelves/Probability_Theory/...

Re: I've stopped using box plots (2021)

#233

Post by crazygringo »

blueflow wrote: post #224
crazygringo wrote: post #190

Earlier quoted context omitted.

> as the 0 and 100 percentile "whiskers" would be at plus/minus infinity The point is not to plot an ideal Gaussian, the point is to plot the data. In real life the whiskers are the actual minimum and maximum values observed.

> In real life the whiskers are the actual minimum and maximum values observed. Look at this: https://upload.wikimedia.org/wikipedia/commons/1/1a/Boxplot_... 0.7% of all values are outside the whiskers.

There are two standard ways of doing box plots. One is miniums and maximums, the other is the 1.5 IQR method.

The very Wikipedia article your image comes from explains this:

https://en.wikipedia.org/wiki/Box_plot#Whiskers

Re: I've stopped using box plots (2021)

#234

Post by seanhunter »

kazinator wrote: post #229
rcxdude wrote: post #40

Earlier quoted context omitted.

Nah, there's nothing in a box plot which assumes a bell-shape. It does, however just visualise the parameters which reasonably well characterize a smooth single-mode distribution regardless of the underlying distribution. So it's a valid criticism of using box plots, especially when the alternatives can just as well visualise a bell-shaped distribution, as well as showing when it is not.

The problem is that the four quantile groups contain equal numbers of items, but are not represented by equal areas, even if we replace the whiskers with a bar of the same width as the box. The bottom whisker contains 25% of the data, yet is just a thin line, which can furthermore be arbitrarily short. It really is a dumb visual presentation. The only way to use it is to recover the five parameters from it, and then…

> The only way to use it is to recover the five parameters from it, and then stop looking at it.

...which is its intended use case since Tukey invented it as a way of visualising the "5 number summary". I think part of his criteria were that it should be easy to make by hand which is clearly no longer a consideration so there are plenty of reasons to just do something else most of the time these days.

Re: I've stopped using box plots (2021)

#235

Post by crazygringo »

theamk wrote: post #208
crazygringo wrote: post #189

Earlier quoted context omitted.

> Just plot all the bloody data and be done with it Well no, because you can compare the datasets by eye and say questionable qualitative things about them, but you can't make definitively true quantitative statements about them. Show me two plots of data points and I can show you two people who will in good faith argue over which one has the higher mean or higher median or higher variance. Because you often can't te…

But there are much better ways to do this than box plots! Lots of CS papers use CDF and it's great and very informative once you get used to it (although you do need to get used to them). You can have violin plots with all the box plots elements and more. Even if you want to restrict yourself to quartiles, author's design concepts with narrow/wide bars makes much more visual sense, and still convey exactly the same i…

It depends on the purpose.

CDF plots are great for plotting a single distributions, but contain way too much information if you want to plot 6 distributions next to each other for easy comparison.

Violin plots are interesting but also quite complicated, since you have to arbitrarily choose a kernel shape and this artificial smoothing can make it look like you have much more data than you really do.

I really don't like the author's "alternative designs" because I think they're even more open to misinterpretation than box plots. It's hard to judge though, because the central problem is that the author is trying to represent a bimodal distribution, and shouldn't be using box plots or the 2 "alternative designs" for that.

Re: I've stopped using box plots (2021)

#238

Post by chipdart »

sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

> Box plots don't make anything bell shaped (...), they assume that your data follows a bell/gaussian shape.

Not true. Box plots represent low and and high quantiles independently and support the representation of outliers.

This alone is enough to make it clear for everyone that they don't require a distribution to be symmetric, let alone bell shape.

Re: I've stopped using box plots (2021)

#240

Post by sanderjd »

seanhunter wrote: post #212
sanderjd wrote: post #174

Earlier quoted context omitted.

Could you summarize the criticisms in this (pretty long) video, and what she is proposing as a better alternative (beanplots? or is she criticizing those too?)? I couldn't figure it out from perusing the transcript. I think it's useful to be able to compare the approximate shapes of histograms during exploratory data analysis. Is the thesis of this criticism that this isn't actually a useful thing to do, or that viol…

The summary is she is saying you almost always want to show one of two things (and not both): 1) To show the distribution, in which case just the histogram arranged horizontally in the traditional fashion is far better than a violin plot with 2 copies of the histogram vertically and some extra quartile stuff tacked on, especially since lots of standard libraries to do violin plots do kde with very extreme smoothing s…

This is helpful!

Is there a different name for the version of this that doesn't include the summary statistics on the same graph? I think seeing the distributions at different x-axis values (in my work, nearly always in a time series), but including the summary statistics is not as important and I agree that it's noisy.

Post reply on HN