Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I've stopped using box plots (2021)

nightingaledvs.com

Post reply on HN
Print view View original

211–220 of 258 posts

Re: I've stopped using box plots (2021)

#211

Post by fjkdlsjflkds »

lolc wrote: post #184
fjkdlsjflkds wrote: post #153

Earlier quoted context omitted.

Note that "not knowing how to correctly interpret a boxplot" is not equivalent to "boxplots are useless".

If people like me are in the audience, they might be worse than useless.

Sure. But if someone is using, for example, a notched boxplot to quickly evaluate differences in medians (i.e., they know how to correctly interpret a boxplot), it can still be a useful plot that conveys specific information that you would otherwise not get when looking at a violin plot, histogram, kernel density estimate or a strip plot.

My point, again, was: just because a boxplot is not useful to some people, doesn't mean that it is not a useful plot (particularly when augmented with a rugplot or a strip plot). Plots are not just used to convey information to others: they are also a useful tool in exploratory data analysis.

Notice that you can also apply the same critique to almost any plot: some people don't know how to interpret a violin plot (or kernel density estimate plot) correctly... does that make them useless?

The main advantage of a boxplot is that it is parameter-free (unlike histograms, violin plots and kernel density plots) and quickly conveys very specific information (median, range, quantiles, confidence interval for the median) that other types of plot usually don't.

Re: I've stopped using box plots (2021)

#212

Post by seanhunter »

sanderjd wrote: post #174
j_bum wrote: post #146

Earlier quoted context omitted.

I disagree about violin plots being better. Here is a great rant (borderline lecture) from Angela Collier on why they aren’t [0] [0] https://youtu.be/_0QMKFzW9fw?si=86mRAZRnFCBfSzw0

Could you summarize the criticisms in this (pretty long) video, and what she is proposing as a better alternative (beanplots? or is she criticizing those too?)? I couldn't figure it out from perusing the transcript. I think it's useful to be able to compare the approximate shapes of histograms during exploratory data analysis. Is the thesis of this criticism that this isn't actually a useful thing to do, or that viol…

The summary is she is saying you almost always want to show one of two things (and not both):

1) To show the distribution, in which case just the histogram arranged horizontally in the traditional fashion is far better than a violin plot with 2 copies of the histogram vertically and some extra quartile stuff tacked on, especially since lots of standard libraries to do violin plots do kde with very extreme smoothing so the distribution they show can be very misleading as to the real empirical distribution.

2) To highlight the summary statistics (quartiles and median) in which case just the boxplot is better because generally these are hard to read on a violin plot

In case #1 this is usually because the distribution differs significantly from a Gaussian in some interesting way that would make a boxplot irrelevant or misleading. (eg it is bimodal or multimodal).

In case #2 this is usually because the distribution is Gaussian (or otherwise standard) and you want to compare it with other standard distributions. You don't need all the information in the histogram and to include it all would obscure the important point(s) you're trying to make about the median and quartiles. What is considered standard is going to depend a lot on the domain, audience and subject matter. In her case, she's an astrophysicist, so if you're looking at say red shift data from some observation, other astrophysicists will know the distribution you would expect to get from that sort of observation for example.

That video is basically a summary of all the conversation attached to this article in some ways.

Re: I've stopped using box plots (2021)

#213

Post by seanhunter »

klysm wrote: post #161
frodo8sam wrote: post #33

Earlier quoted context omitted.

I'll take a plain histogram/kde plot every day of the week over those damn violin plots. I think box plots are quite usefull as they are easy to read but only if you trust the author has actually looked at the histogram. And you can typically not trust the author to have done that.

A violin plot is literally just a KDE sideways.

It also has a box plot tacked on because "why not"?

Re: I've stopped using box plots (2021)

#215

Post by callalex »

kaitai wrote: post #171
iainmerrick wrote: post #130

Lots of people defending box plots here -- a lot more than I expected! What I don't see is anyone saying "box plots are useful because they're the best kind of chart for [specific use case]". I can't off-hand think of any situation where I'd rather see a box plot than a strip plot or violin plot. When and why would you want to summarise the data so coarsely and visualize it so un-intuitively?

I deal with a lot of business people who have processes that rely on 15th/85th percentile, or 25th/75th percentile. They want to see the median, the low/high percentiles, the max/min or outliers, and they don't want to see all the data points jittered in between. It's just overwhelming extraneous information. They in fact like tables with those numbers written down, but they want to compare ten different (time series…

Who drew those boundaries at 15/85? What makes those boundaries useful or correct?

Re: I've stopped using box plots (2021)

#216

Post by blueflow »

Falkon1313 wrote: post #201
blueflow wrote: post #172

Earlier quoted context omitted.

On second thought, this method makes the outer brackets / whiskers pretty much useless since their position is determined by the largest outliers, which is quite much random.

That's not how they're drawn. Outliers (More than 1.5 times the interquartile range outside the 1st/3rd quartile) are plotted as dots beyond the whiskers. The whiskers go at Q1-1.5×IQR and Q3+1.5×IQR.

Better is! Look what i was replying to.

Re: I've stopped using box plots (2021)

#217

Post by blueflow »

thaumasiotes wrote: post #203
blueflow wrote: post #124

Earlier quoted context omitted.

If the method how the plot boxes are calculated is not clear (this thread references at least two different methods), you'll need to explicitly write it down which methods you did use.

> this thread references at least two different methods No, as the sidethread comment notes, there is only one way you can compute quartiles. You seem to be arguing that the correct thing to do is to impute them, and that calculating them is such a deviant practice that it would need to be specially remarked on.

Isn't this what i was saying from the beginning?

  Box plots are made for visualizing generalized normal distributions and nothing else.
And now people in this thread argue you can calculate them from something else. Not sure if you are replying to the right post.

Re: I've stopped using box plots (2021)

#218

Post by sigmoid10 »

fnordpiglet wrote: post #170
sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

Sorry I don’t understand. The central limit theorem describe the distribution of the sample means from a population. It describes the distribution of the mean, not the distribution of the population itself. The shape of the distribution of the sample mean isn’t super interesting when you’re interested in the distribution of the samples themselves as a proxy for a population. So I’m not sure I understand your assertio…

I't appears you don't understand the central limit theorem fully. You gave the definition you find in textbooks, but you don't see how it applies to real world measurements and already explains your question. I can only recommend to visit a university level statistics course at this point. Maybe you will understand when you actually deal with some real data. Then you will indeed see its consequences pop up everywhere. The issue (also for the blog author) is that it is often implicitly assumed. It is one of many common pitfalls in statistics. You should also learn what the difference is between a poisson and a gaussian distribution. They may look similar, but there is a drastic difference in their definition and they are used in very different circumstances.

Re: I've stopped using box plots (2021)

#220

Post by s1artibartfast »

iainmerrick wrote: post #130

Lots of people defending box plots here -- a lot more than I expected! What I don't see is anyone saying "box plots are useful because they're the best kind of chart for [specific use case]". I can't off-hand think of any situation where I'd rather see a box plot than a strip plot or violin plot. When and why would you want to summarise the data so coarsely and visualize it so un-intuitively?

>When and why would you want to summarise the data so coarsely and visualize it so un-intuitively?

Sometimes less is more; Box plots are specifically good for showing and comparing quartiles.

If you want to compare several groups and care about gross differences, they are an excellent tool. They are an excellent to when you believe the data is normal and think the histogram is misleading. they are also great if you think the data isnt normal but care about quartiles.

Any time you would be happy with a table of the 5 datapoints (min, max, median, 25th, and 75th percentiles), box plots a great tool for graphic comparison.

Post reply on HN