Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I've stopped using box plots (2021)

nightingaledvs.com

Post reply on HN
Print view View original

191–200 of 258 posts

Re: I've stopped using box plots (2021)

#191

Post by crazygringo »

cjk2 wrote: post #19
blueflow wrote: post #14

Earlier quoted context omitted.

This should be the topmost comment. Box plots are made for visualizing generalized normal distributions and nothing else. Edited to preempt nitpick.

This is also wrong. Gaussian curves are symmetric. Box plots do not have to be. In fact representing skew in a batch is one of the fundamental purposes of them.

But representing skew is precisely to show how "off" from a Guassian it is.

Because real data is never perfectly Guassian, or perfectly anything.

But the idea of a box plot is that it's for data which is in theory Gaussian or a similar unimodal kind of bell-shaped curve.

Then you can look at the box plot and see if it actually is -- are the two boxes roughly equal-sized? Are the lines a bit longer than the boxes but not insanely so?

Re: I've stopped using box plots (2021)

#192

Post by pictureofabear »

crazygringo wrote: post #173
sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

Yes. A lot of people here are commenting that no, technically box plots don't assume any distribution. And I mean, technically you can ride from NYC to SF in a lawnmower. But I completely agree that box plots shouldn't ever be used for anything but unimodal distributions similar enough to a bell/gaussian distribution. All of the criticism of the article seems to be that they're misleading when the distribution is not…

This article is very click-baity.

Boxplots are a single tool for data analysis. They do not apply in every situation, nor do any other tools. The same goes for pie charts, which are constantly being accused of always distorting data. Pie charts, like box plots, have their place.

Re: I've stopped using box plots (2021)

#193

Post by afiori »

crazygringo wrote: post #188
afiori wrote: post #162

Earlier quoted context omitted.

Quartiles are relevant for almost any distribution

If by "almost any" you mean "unimodal". Quartiles are not relevant, i.e. can be highly misleading, for a bimodal distribution or beyond...

they are misleading if you assume unimodality, but are always relevant. If you care about how many modes there are then likely you would prefer deciles or centiles.

But even in the first image of the article the fact that two quartiles are close together means that there some density peak around there.

I agree with the author that box plots are not good plots, but quartiles/deciles/medians are useful even for multimodal distributions

Re: I've stopped using box plots (2021)

#194

Post by sanderjd »

interroboink wrote: post #181
sanderjd wrote: post #174

Earlier quoted context omitted.

Could you summarize the criticisms in this (pretty long) video, and what she is proposing as a better alternative (beanplots? or is she criticizing those too?)? I couldn't figure it out from perusing the transcript. I think it's useful to be able to compare the approximate shapes of histograms during exploratory data analysis. Is the thesis of this criticism that this isn't actually a useful thing to do, or that viol…

Her criticisms of violin plots seem to be (1) they combine histogram-style information with box-plot-style information, when you generally would only want one or the other [ie: don't use boxplot for bimodal, don't use histogram when boxplot suffices], (2) The histogram-style information is not comparable between blobs of data, since they're not visually aligned, have no tick marks, etc — a plain histogram is better f…

Thanks for this summary! I definitely hadn't seen the point about comparability between blobs of data because of the alignment. But that really seems like an odd point to me, as I almost entirely see / use these with time series data, where pretty much the whole point is to compare the evolution of the values over time using their "vertical" location, with a was to see the shape of a distribution of values at each point in time, at a glance.

Re: I've stopped using box plots (2021)

#195

Post by SebastianKra »

sanderjd wrote: post #174
j_bum wrote: post #146

Earlier quoted context omitted.

I disagree about violin plots being better. Here is a great rant (borderline lecture) from Angela Collier on why they aren’t [0] [0] https://youtu.be/_0QMKFzW9fw?si=86mRAZRnFCBfSzw0

Could you summarize the criticisms in this (pretty long) video, and what she is proposing as a better alternative (beanplots? or is she criticizing those too?)? I couldn't figure it out from perusing the transcript. I think it's useful to be able to compare the approximate shapes of histograms during exploratory data analysis. Is the thesis of this criticism that this isn't actually a useful thing to do, or that viol…

Her argument that convinced me, is that the same result can always be better represented with multiple histograms - z-stacked, side-by-side, 3D or ridgeline-plots (ridgeline plots look awesome). Check out her examples at 21:11.

Compared to these alternatives, violin plots are comically bad.

Re: I've stopped using box plots (2021)

#196

Post by Aachen »

sanderjd wrote: post #174
j_bum wrote: post #146

Earlier quoted context omitted.

I disagree about violin plots being better. Here is a great rant (borderline lecture) from Angela Collier on why they aren’t [0] [0] https://youtu.be/_0QMKFzW9fw?si=86mRAZRnFCBfSzw0

Could you summarize the criticisms in this (pretty long) video, and what she is proposing as a better alternative (beanplots? or is she criticizing those too?)? I couldn't figure it out from perusing the transcript. I think it's useful to be able to compare the approximate shapes of histograms during exploratory data analysis. Is the thesis of this criticism that this isn't actually a useful thing to do, or that viol…

The two other replies are her main point(s), but the video also spends some time on another issue that she labels as minor but I found interesting to hear the perspective on. I'll try to do it justice:

They look like vulvas. We're all adults, it's not a problem typically, but given that it's an aesthetic choice (noticing how half of the chart conveys the same info without this property), why? And it does come up, like if someone does make a joke about it, a room full of typically only well-meaning men will now look to her if she's comfortable with the joke and, what was okay before, now turns into a feeling of being singled out and outside the rest of the group

Re: I've stopped using box plots (2021)

#198

Post by nequo »

davidguetta wrote: post #183
conformist wrote: post #99

Earlier quoted context omitted.

In the mathematical sense this is clearly not true - it’s easy to come up with a smooth single mode distribution that doesn’t look like a bell.

The bell curve IS the smooth single mode with LOWEST ENTROPY.

Do you mean greatest entropy? Not if the support is, for example, the positive reals.

Re: I've stopped using box plots (2021)

#199

Post by ColFrancis »

blueflow wrote: post #128
munch117 wrote: post #123

Earlier quoted context omitted.

Perhaps I expressed myself poorly, and left room for misunderstanding, because I cannot possible imagine that we have any real disagreement on how to compute quartiles. Any set of numbers I give you, you can compute quartiles for it. There is no algorithm for doing that that breaks down if the numbers don't follow a normal distribution.

Look at this SVG from wikipedia: https://upload.wikimedia.org/wikipedia/commons/1/1a/Boxplot_... When you calculate the box plot using normal distribution parameters, the outliers are outside the outer bracket. If you split the dataset into 4 equal parts, the bracket will be larger because the outliers are still inside it. The methodologies are not equal. This thread is the first time i heard people do the "split dat…

For what it's worth, you've convinced me that my beloved box plots need to be explained if I want to use them again.

The SVG you've provided clearly shows that the box plot splits the data in 4. The interquartile range (IQR) is clearly marked and it even has a comparison for what the standard deviation (variance) measure would be.

Secondly, if the data truly came from a normal distribution, there are no outliers. Outliers are data points which cannot be explained by the model and need to be removed. Unless you have a good reason to exclude the data points they should be included. This is why I like the IQR and the median, they are not swayed by a few wide valued data points. The 1.5*IQR rejection filter I think is lazy and unjustified. Happy to discuss this point further as it is a bug bear of mine.

Re: I've stopped using box plots (2021)

#200

Post by catlifeonmars »

medstrom wrote: post #116
frodo8sam wrote: post #33

Earlier quoted context omitted.

I'll take a plain histogram/kde plot every day of the week over those damn violin plots. I think box plots are quite usefull as they are easy to read but only if you trust the author has actually looked at the histogram. And you can typically not trust the author to have done that.

Perhaps you'd find the half-violin plot more readable? Seems there's a whole world of all-in-one "raincloud plots" that integrates them, like the lower infographic here: https://raw.githubusercontent.com/Z3tt/TidyTuesday/main/plot... You can even make 'em show histograms: https://miro.medium.com/v2/1*J3Q4JKXa9WwJHtNaXRu-kQ.jpeg

For the latter, it looks like the histogram is probably sufficient. The violin plot just adds extra visual noise.
Post reply on HN