Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I've stopped using box plots (2021)

nightingaledvs.com

Post reply on HN
Print view View original

101–110 of 258 posts

Re: I've stopped using box plots (2021)

#101

Post by munch117 »

blueflow wrote: post #14
sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

This should be the topmost comment. Box plots are made for visualizing generalized normal distributions and nothing else. Edited to preempt nitpick.

But is that what they're actually used for?

The data has been reduced to three numbers, throwing away most of the information that you would need to assess whether the distribution is gaussian or not. If it's not, how will you ever know?

Re: I've stopped using box plots (2021)

#102

Post by munch117 »

blueflow wrote: post #100
pocketsand wrote: post #92

Earlier quoted context omitted.

Why? They’re non-parametric and make zero assumptions of normality.

How else would you calculate the quartiles to render the boxes?

Count data points in each quartile. You can do that for any sortable data, independent of distribution.

Re: I've stopped using box plots (2021)

#103

Post by blueflow »

munch117 wrote: post #102
blueflow wrote: post #100

Earlier quoted context omitted.

How else would you calculate the quartiles to render the boxes?

Count data points in each quartile. You can do that for any sortable data, independent of distribution.

If you do that in your paper, you better write next to the graph that you did that.

Re: I've stopped using box plots (2021)

#104

Post by smcin »

rcxdude wrote: post #41
treflop wrote: post #34

Earlier quoted context omitted.

I agree. The author simply used the wrong chart. The author's example has a bimodal distribution (TWO peaks) and chooses a type of chart that has ONE peak (a box plot). A little baffling tbh.

Well, to start with, how would you determine that about your distribution in the first place? And if that works well enough, why use a box plot afterwards?

Simple, use a histogram.

The author's first histogram clearly shows most of the distribution lies in [20,100), then the [10,20) bin is empty but the [0,10) bin is quite full. Hence, that's not a single-mode distribution. It has two modes, one around [50,60) and the other in [0,10).

Re: I've stopped using box plots (2021)

#105

Post by thaumasiotes »

conformist wrote: post #99
quietbritishjim wrote: post #88

Earlier quoted context omitted.

> smooth single-mode distribution That IS a bell curve. While it's true that the Guassian distribution is often called a bell curve or even "the" bell curve, a non-Guassian single mode distribution is still absolutely bell shaped in a general sense. So, although you started your comment with "nah", you're actually in agreement with the content you replied to.

In the mathematical sense this is clearly not true - it’s easy to come up with a smooth single mode distribution that doesn’t look like a bell.

Is it? How?

You could include a lot of little bells far from the single mode, but that's reading a little too much into the literal meaning of "single mode" - a "bimodal" distribution isn't one where the two most common values are both modes. It's one where there are two distinct local maxima.

The tails to the left and right must asymptotically approach zero (or you don't have a smooth distribution, because you have discontinuities somewhere), and if there's just one local maximum, your curve will look like a bell.

Re: I've stopped using box plots (2021)

#106

Post by ncruces »

lkdfjlkdfjlg wrote: post #96
ncruces wrote: post #54

Earlier quoted context omitted.

> You don't know where the rest are. Of course you do: they're in the whiskers; half in each whisker. That's the entire point of the picture, BTW.

You're right. I guess that's not the author's mistake then. His mistake is assuming "the whisker is small, therefore it has a small number of datapoints".

That's not his mistake. He knows this, but repeatedly failed to convey this to others.

That's like the entire point of the post: they're hard to teach to others (they're unintuitive) and there are better (more intuitive) alternatives.

I dunno if I agree, but it's ironic that this thread started with a poster complaining about the author's bad intuition, while apparently managing to not have a good grasp of box plots themselves.

Re: I've stopped using box plots (2021)

#107

Post by thaumasiotes »

treflop wrote: post #34
sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

I agree. The author simply used the wrong chart. The author's example has a bimodal distribution (TWO peaks) and chooses a type of chart that has ONE peak (a box plot). A little baffling tbh.

> and chooses a type of chart that has ONE peak (a box plot)

Huh? A box plot doesn't have any peaks. A box plot is a histogram subject to the constraint that every bar in the histogram is equally tall. There can never be more than zero peaks.

Re: I've stopped using box plots (2021)

#108

Post by munch117 »

cjk2 wrote: post #93
munch117 wrote: post #91

Earlier quoted context omitted.

Isn't that the whole point? That the graph type is very easy to misunderstand. If you are right, and not even a professional data visualization consultant properly understands the graph, then who will?

Some of us are perfectly qualified to understand them and the nuances.

A plot that requires the reader to be perfectly qualified is a bad plot.

Re: I've stopped using box plots (2021)

#109

Post by sloowm »

cjk2 wrote: post #10

No you shouldn’t stop using box plots. You should use them for when they are appropriate - showing location and spread. And not shape! There’s absolutely no information on modality or distribution presented past quartiles and limits. They are mostly useful for comparing batches not analysing an individual batch. The author doesn’t know what they are talking about and is telling people as if they do. If he read any of…

You absolutely should stop using box plots. The only reason to use them is because you have to draw a representation by hand and do not have access to a computer.

A box plot is a data compression technique for compression by hand. There are now better automated techniques that both preserve data quality and visual quality better.

Re: I've stopped using box plots (2021)

#110

Post by gradstudent »

sigmoid10 wrote: post #32
Beldin wrote: post #24

Earlier quoted context omitted.

> Box plots [...] assume that your data follows a bell/gaussian shape. Not sure how to square that with this statement on Wikipedia's page on box plots: Box plots are non-parametric: they display variation in samples of a statistical population without making any assumptions of the underlying statistical distribution[3]

If you want to see why that is not fully correct you should read the article. For a box plot you need to calculate mean, variance and certain percentiles. These values don't make sense if your distribution does not follow a certain shape (because these values unambiguously define such a shape). See the examples in the article for what happens if you still try to use them in those cases. You can still extract the valu…

> because these values unambiguously define such a shape

I think this is a misunderstanding, and I think it is shared by the author of the article. Boxpolots show ranges. That's it.

Post reply on HN