Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I've stopped using box plots (2021)

nightingaledvs.com

Post reply on HN
Print view View original

201–210 of 258 posts

Re: I've stopped using box plots (2021)

#201

Post by Falkon1313 »

blueflow wrote: post #172
munch117 wrote: post #102

Earlier quoted context omitted.

Count data points in each quartile. You can do that for any sortable data, independent of distribution.

On second thought, this method makes the outer brackets / whiskers pretty much useless since their position is determined by the largest outliers, which is quite much random.

That's not how they're drawn. Outliers (More than 1.5 times the interquartile range outside the 1st/3rd quartile) are plotted as dots beyond the whiskers. The whiskers go at Q1-1.5×IQR and Q3+1.5×IQR.

Re: I've stopped using box plots (2021)

#202

Post by Falkon1313 »

I was not entirely convinced by the article, being used to box plots myself for several decades. I've used them in school, college, and at work.

But after having read these comments, it really drives home his point that you can get a room full of lots of very smart people who all know what they're talking about, and they'll all disagree about the understanding and interpretation of box plots.

It's a little surprising, but the evidence in these threads pretty much cinches the argument for me.

Re: I've stopped using box plots (2021)

#203

Post by thaumasiotes »

blueflow wrote: post #124
thaumasiotes wrote: post #113

Earlier quoted context omitted.

Arguing that nobody who might be professionally expected to look at a box plot can be reasonably expected to understand how box plots are defined doesn't make a compelling case that using them is a good idea.

If the method how the plot boxes are calculated is not clear (this thread references at least two different methods), you'll need to explicitly write it down which methods you did use.

> this thread references at least two different methods

No, as the sidethread comment notes, there is only one way you can compute quartiles. You seem to be arguing that the correct thing to do is to impute them, and that calculating them is such a deviant practice that it would need to be specially remarked on.

Re: I've stopped using box plots (2021)

#204

Post by hoosieree »

crazygringo wrote: post #173
sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

Yes. A lot of people here are commenting that no, technically box plots don't assume any distribution. And I mean, technically you can ride from NYC to SF in a lawnmower. But I completely agree that box plots shouldn't ever be used for anything but unimodal distributions similar enough to a bell/gaussian distribution. All of the criticism of the article seems to be that they're misleading when the distribution is not…

Murphy's law for data viz:

If a plot can mislead, it will.

Re: I've stopped using box plots (2021)

#205

Post by nkrisc »

cjk2 wrote: post #117
magnio wrote: post #115

Earlier quoted context omitted.

You are looking at this as a technical problem, where box plot is a compact visual representation of variance and outliers that is perfectly perfunctory as it is cromulent. The author is approaching this as a human problem. Plots are not made for machines, they are for people to read, and the author specifically wants as many people can read and parse plots easily as possible. As lamentable as math education might be…

So your approach and the author’s is to dumb a technical measure down to a level where the observer doesn’t need to understand what they are looking at. Well that explains the entire data visualisation and dashboard consultancy nicely. How does anyone rationalise the information they have if they don’t make an effort to understand it. Or how can they even select a visualisation method or comparison method. We are tru…

Do you want to be right, or do you want to be understood?

You can’t control what other people do. You can try to meet them where they are, or hope they’ll catch up with you. Hopefully it’s not your problem if they fail to.

Re: I've stopped using box plots (2021)

#206

Post by theamk »

crazygringo wrote: post #173
sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

Yes. A lot of people here are commenting that no, technically box plots don't assume any distribution. And I mean, technically you can ride from NYC to SF in a lawnmower. But I completely agree that box plots shouldn't ever be used for anything but unimodal distributions similar enough to a bell/gaussian distribution. All of the criticism of the article seems to be that they're misleading when the distribution is not…

Well, how do you readers know if your distribution is bell/gaussian? Sure, sometimes you plot means of large samples, and then it is true by construction; but a lot of time people use box plots when there is no intrinsic reasons for data to be gaussian. Like most experimental papers.

Or take the first example from wikipedia page on box plot [0]: "Box plot of data from the Michelson experiment", which is just 20 points per run. Would I want to see this in the paper? No please. There is no evidence that the experimental data is gaussian (or even single-modal). Or further down that page, "A series of hourly temperatures" - why would one box-plot it either?

And even if you claim your data is gaussian by construction, maybe because you surveyed lots of people - I still want to see the evidence, as it's pretty simple to make experimental mistakes that turns data non-gaussian (say you only surveyed two neighborhoods with very different properties)

In other words, the domain where box plots are sufficient is very small. Most publications should never use them.

[0] https://en.wikipedia.org/wiki/Box_plot

Re: I've stopped using box plots (2021)

#208

Post by theamk »

crazygringo wrote: post #189
WWWWH wrote: post #179

Earlier quoted context omitted.

Yes, exactly! Just plot all the bloody data and be done with it. No one is doing this by hand anymore so it is no extra work. To my mind, if you have a genuine EDA attitude you plot it all.

> Just plot all the bloody data and be done with it Well no, because you can compare the datasets by eye and say questionable qualitative things about them, but you can't make definitively true quantitative statements about them. Show me two plots of data points and I can show you two people who will in good faith argue over which one has the higher mean or higher median or higher variance. Because you often can't te…

But there are much better ways to do this than box plots! Lots of CS papers use CDF and it's great and very informative once you get used to it (although you do need to get used to them). You can have violin plots with all the box plots elements and more. Even if you want to restrict yourself to quartiles, author's design concepts with narrow/wide bars makes much more visual sense, and still convey exactly the same information as box plots.

Re: I've stopped using box plots (2021)

#209

Post by __mharrison__ »

I've resorted to just teaching four plot types when I teach visualization.

- Bar

- Scatter

- Line

- Histogram

You can tell 90% of your stories with these plots. (If you pay attention to professional viz groups, Economist, NY Times, etc, they use these.)

Don't waste your time with other plots unless you have mastered these. When you master these, you will realize you don't need other charts.

Re: I've stopped using box plots (2021)

#210

Post by mannykannot »

crazygringo wrote: post #173
sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

Yes. A lot of people here are commenting that no, technically box plots don't assume any distribution. And I mean, technically you can ride from NYC to SF in a lawnmower. But I completely agree that box plots shouldn't ever be used for anything but unimodal distributions similar enough to a bell/gaussian distribution. All of the criticism of the article seems to be that they're misleading when the distribution is not…

The article's full argument seems to be that there are alternatives which are applicable where box plots are not and, at least in most cases, better where they are (there is a tacit (IIRC) subtext of "given that we're using software to do the plotting.")

This is debatable, but noting that box plots are satisfactory for unimodal gaussian-ish distributions is not a very persuasive response.

Post reply on HN