Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I've stopped using box plots (2021)

nightingaledvs.com

Post reply on HN
Print view View original

161–170 of 258 posts

Re: I've stopped using box plots (2021)

#161

Post by klysm »

frodo8sam wrote: post #33
mkl wrote: post #21

The only advantage box plots had is that they can be drawn by hand. Now that computers are ubiquitous this is no longer valuable. Violin plots and bee swarm plots are better. Jittered strip plots can be okay if you're careful to avoid saturation (or more points added in the saturated region will disappear as they can't make it any darker).

I'll take a plain histogram/kde plot every day of the week over those damn violin plots. I think box plots are quite usefull as they are easy to read but only if you trust the author has actually looked at the histogram. And you can typically not trust the author to have done that.

A violin plot is literally just a KDE sideways.

Re: I've stopped using box plots (2021)

#162

Post by afiori »

kamma4434 wrote: post #112
IanCal wrote: post #56

Earlier quoted context omitted.

> they assume that your data follows a bell/gaussian shape No they don't. They show quartiles mostly, and don't assume symmetry or any parameters of a gaussian.

What you say is technically correct, but in the sense where you can put rat poison in One of those ceramic cookie jars they sell in houseware shops. There is nothing wrong in doing it, but it may lead to interesting failure modes Because someone can have implicit assumptions about what’s in there.

Quartiles are relevant for almost any distribution

Re: I've stopped using box plots (2021)

#163

Post by weebull »

DonsDiscountGas wrote: post #149
iainmerrick wrote: post #130

Lots of people defending box plots here -- a lot more than I expected! What I don't see is anyone saying "box plots are useful because they're the best kind of chart for [specific use case]". I can't off-hand think of any situation where I'd rather see a box plot than a strip plot or violin plot. When and why would you want to summarise the data so coarsely and visualize it so un-intuitively?

Violin plots are massively overhyped, IMHO. If your data is simple and unimodal, use a boxplot. If the distribution is more complicated and you need some detail, use a histogram or a ridge plot. Violin plots are never the best option; they're curvy so a little more pretty but don't do a good job of conveying information.

> If your data is simple and unimodal, use a boxplot.

How is the reader to know you've used the right plot? How are they to know that you haven't hidden a bimodel dataset behind a box plot because it makes your conclusions easier?

> If the distribution is more complicated and you need some detail, use a histogram or a ridge plot. Violin plots are never the best option; they're curvy so a little more pretty but don't do a good job of conveying information.

They are just multiple, non-overlapping histograms plotted next to each other. They allow you to compare distributions without them getting in the way of each other.

I can understand if it's the fitted PDF that you think hides the original data. That is unnecessary IMHO.

Re: I've stopped using box plots (2021)

#164

Post by weebull »

lkdfjlkdfjlg wrote: post #135
iainmerrick wrote: post #130

Lots of people defending box plots here -- a lot more than I expected! What I don't see is anyone saying "box plots are useful because they're the best kind of chart for [specific use case]". I can't off-hand think of any situation where I'd rather see a box plot than a strip plot or violin plot. When and why would you want to summarise the data so coarsely and visualize it so un-intuitively?

> What I don't see is anyone saying "box plots are useful because they're the best kind of chart for [specific use case]". Box plots are useful because they're the best kind of chart for when I have multiple populations and I want to quickly glance whether it's reasonable to assume that the populations have the same median, or not (you do that comparing not just the medians of the populations but also the shaded area…

If you're only comparing medians, then just plot the medians. Why a box plot with the quartiles?

Re: I've stopped using box plots (2021)

#165

Post by kibwen »

cjk2 wrote: post #117
magnio wrote: post #115

Earlier quoted context omitted.

You are looking at this as a technical problem, where box plot is a compact visual representation of variance and outliers that is perfectly perfunctory as it is cromulent. The author is approaching this as a human problem. Plots are not made for machines, they are for people to read, and the author specifically wants as many people can read and parse plots easily as possible. As lamentable as math education might be…

So your approach and the author’s is to dumb a technical measure down to a level where the observer doesn’t need to understand what they are looking at. Well that explains the entire data visualisation and dashboard consultancy nicely. How does anyone rationalise the information they have if they don’t make an effort to understand it. Or how can they even select a visualisation method or comparison method. We are tru…

> So your approach and the author’s is to dumb a technical measure down to a level where the observer doesn’t need to understand what they are looking at.

this is precisely why i don't bother with capitalization in my sentences.

in fact even punctuation isnt necessary i dont see why i should dumb down my explanations for people who arent going to make an effort to understand them

actuallyevenspacesaresimplyredundantandasufficientlysmartreadershouldjustunderstandmymeaningwithoutmeneedingtodelineatemywordswhataretheyachildifthiswasgoodenoughfortheancientromansthenitsgoodenoughforme

hckvnvwlsrrdndntndfnynsysthrwsthnmycnclsnsthtthrbrnsrnsffcntlylrgtcmprhndmygns

Re: I've stopped using box plots (2021)

#166

Post by ubercow13 »

cjk2 wrote: post #95
ohmyiv wrote: post #89

Earlier quoted context omitted.

> The author is way out of their depth and should retract the article and take a formal, accredited statistics course. Maybe you should learn about the author before you make such assumptions. I find it hilarious you think he should take statistics courses when he teaches data visualization workshops to places like NASA, IRS, and the UN. I'm done with this thread. Such a joke.

Oh I know the author. Just because you’re high profile in the data viz industry doesn’t mean you should be commenting on statistics especially with such a clear misunderstanding going on. Some of us are definitely more qualified to speak on these matters and we still don’t think we’re qualified to teach it.

If box plots require an formal and accredited statistics course to understand, but as you mention they are taught to 15 year olds (presumably incorrectly) in school and used by people with power making decisions that affect everyone in organisations such as the UN and NASA, then even if the author is unqualified it seems their point is 'accidentally' correct. No one should be using these plots except extremely smart and trained people who do know how to read them, as it could have serious negative consequences.

Re: I've stopped using box plots (2021)

#167

Post by parpfish »

inciampati wrote: post #155
DonsDiscountGas wrote: post #149

Earlier quoted context omitted.

Violin plots are massively overhyped, IMHO. If your data is simple and unimodal, use a boxplot. If the distribution is more complicated and you need some detail, use a histogram or a ridge plot. Violin plots are never the best option; they're curvy so a little more pretty but don't do a good job of conveying information.

They really help when you're working with huge numbers. It's just a different kind of density plot. A vertical histogram can be nice too. Or you can use color and overlay a few regular old histograms. Go wild.

Overlaid histos can be confusing because people don’t know if they are stacked or overlapped.

One solution is to smooth into a kde and then use transparency to indicate overlap, but that’s introducing more complexity than you want for a quick n dirty first pass

Re: I've stopped using box plots (2021)

#168

Post by jncfhnb »

sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

> people need to understand the basics of statistics before they can use them.

> they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere)

You sir just failed basic statistics

Re: I've stopped using box plots (2021)

#169

Post by jncfhnb »

The author showed jittered strip plots where you plot each point correctly on the y axis and randomly offset the x axis.

These are ok but it’s hard to differentiate the density of points when they’re randomly offset. Try a swarm plot (seaborn) / bee swarm plot (R).

It’s the same concept but the points are strategically placed across the x axis to show the width of the distribution at each point. It generally looks much cleaner.

Re: I've stopped using box plots (2021)

#170

Post by fnordpiglet »

sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

Sorry I don’t understand. The central limit theorem describe the distribution of the sample means from a population. It describes the distribution of the mean, not the distribution of the population itself. The shape of the distribution of the sample mean isn’t super interesting when you’re interested in the distribution of the samples themselves as a proxy for a population. So I’m not sure I understand your assertion. Could you explain more your reasoning? Maybe I’m missing something, but the estimation of the sample mean distribution isn’t the only metric that’s useful, and almost nothing in nature is normally distributed otherwise. Normal distributions are generally a useful assumption mostly because of the analytic form of the Gaussian and our understanding of how to work with it. But that estimation isn’t useful as it might seem. A Poisson distribution is much more common for instance.
Post reply on HN