Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I've stopped using box plots (2021)

nightingaledvs.com

Post reply on HN
Print view View original

61–70 of 258 posts

Re: I've stopped using box plots (2021)

#61

Post by cjk2 »

rcxdude wrote: post #41
treflop wrote: post #34

Earlier quoted context omitted.

I agree. The author simply used the wrong chart. The author's example has a bimodal distribution (TWO peaks) and chooses a type of chart that has ONE peak (a box plot). A little baffling tbh.

Well, to start with, how would you determine that about your distribution in the first place? And if that works well enough, why use a box plot afterwards?

Because it's very hard to rationally compare multimodal batches without single test statistics. And they present five summary figures for each batch, each of which are reasonable metrics to compare batches with.

Re: I've stopped using box plots (2021)

#62

Post by [deleted] »

rcxdude wrote: post #40
sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

Nah, there's nothing in a box plot which assumes a bell-shape. It does, however just visualise the parameters which reasonably well characterize a smooth single-mode distribution regardless of the underlying distribution. So it's a valid criticism of using box plots, especially when the alternatives can just as well visualise a bell-shaped distribution, as well as showing when it is not.

[deleted]

Re: I've stopped using box plots (2021)

#63

Post by SillyUsername »

Angostura wrote: post #45
SillyUsername wrote: post #35

Earlier quoted context omitted.

There are a few advantages (see visualization section here pls http://en.m.wikipedia.org/wiki/Box_plot ) but my main concern is that the problem is not with the diagram, it's with idea that it's somehow faulty. Sometimes you may want to highlight some core representation of data without the distraction of outliers (yes that does mean some people will use it for deliberate misrepresentation). But in this regard it's u…

The article doesn’t really argue that they are “faulty” just that there are better alternatives in the large majority of cases. I think he makes a compelling argument

Fair enough, it was the comment that "better-designed chart types" that caught my eye. "better designed for general use" should have been the context I read it in.

Re: I've stopped using box plots (2021)

#64

Post by nosianu »

SillyUsername wrote: post #13
SillyUsername wrote: post #8

So the diagram should not be used because of an education problem with some audiences? Isn't that a bit like banning cars because some people can't drive? Some diagrams are simply not for mass consumption and this is one, particularly because it is designed to illustrate an interpretation of ranges instead of the direct/linear representation of the raw data. Of course I'd illustrate this fact as a Venn diagram compar…

[flagged]

I did not vote, but I can understand it - because of your first sentence already:

> So the diagram should not be used because of an education problem with some audiences?

You dismiss the problem of education as if that is free - but information is physical and spreading it takes significant amounts of time and energy, brains are what they are and hard to change, so this is so obviously a very significant problem that I don't see a basis for discussion given the context here. A forum such as this is a bad place to talk about very basic assumptions, to be able to have a useful discussion about topics such as this some minimum common understanding needs to be there.

Accepting the reality of how people think and behave is rational. To answer your rhetorical question: Yes! That reason is valid.

We usually only have less than a hundred comments that are useful, many more and most won't ever even see them. If we had to discuss such basics, it would be a huge waste of time, and it would be detrimental to the overall value of the discussion. That includes explaining it to the commenter.

I think it is okay to make such comments less visible. In my view it's less about "punishment" or about annoying the writer, but about letting the other people concentrate on other comments that don't force one into side-tracked discussions about very basic things.

I would suggest that you don't take it personal, we all occasionally are in that same boat.

Re: I've stopped using box plots (2021)

#65

Post by ohmyiv »

cjk2 wrote: post #10

No you shouldn’t stop using box plots. You should use them for when they are appropriate - showing location and spread. And not shape! There’s absolutely no information on modality or distribution presented past quartiles and limits. They are mostly useful for comparing batches not analysing an individual batch. The author doesn’t know what they are talking about and is telling people as if they do. If he read any of…

> No you shouldn’t stop using box plots. You should use them for when they are appropriate

Yes, the author is aware of that. They even stated so:

> Despite making more visual sense than box plots, I still wouldn’t recommend these design concepts or box plots in most situations because…

Seems a few people missed the "in most situations" part. He's saying he stopped using them for whatever reasons because it isn't working for his audience. So as the title suggests, maybe we should all take a look at our use of box plots and see if there are better alternatives.

Also remember who he's talking about when it comes to reading box plots. He's not talking about people who understand box plots. He's talking about others that don't know or understand box plots, which seems to be thousands of people he's had to explain it to, according to him.

Re: I've stopped using box plots (2021)

#66

Post by CuriouslyC »

Box plots are a relic of a time when we couldn't print really nice charts. You can just display the distribution in line like a scrolling oscilloscope/topographic display, or you can do a density plot over time (look at gaussian processes) and overlay shaded regions for important time periods.

Re: I've stopped using box plots (2021)

#67

Post by rzmmm »

ozyschmozy wrote: post #36
sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

> There are very real use cases for box plots, The author argues otherwise, can you give an example of a use case where box plots would be preferable to the alternatives the author suggests?

Often people are interested in exact quantitative statistics like IQR, median, top/bottom deciles which are commonly represented in box plots. The alternatives are visually simpler but they contain less quantitative information.

Re: I've stopped using box plots (2021)

#68

Post by JumpCrisscross »

sigmoid10 wrote: post #32
Beldin wrote: post #24

Earlier quoted context omitted.

> Box plots [...] assume that your data follows a bell/gaussian shape. Not sure how to square that with this statement on Wikipedia's page on box plots: Box plots are non-parametric: they display variation in samples of a statistical population without making any assumptions of the underlying statistical distribution[3]

If you want to see why that is not fully correct you should read the article. For a box plot you need to calculate mean, variance and certain percentiles. These values don't make sense if your distribution does not follow a certain shape (because these values unambiguously define such a shape). See the examples in the article for what happens if you still try to use them in those cases. You can still extract the valu…

> For a box plot you need to calculate mean, variance

Quantiles and medians. (Plus min and max.) Non-parametric.

Re: I've stopped using box plots (2021)

#69

Post by mkl »

frodo8sam wrote: post #33
mkl wrote: post #21

The only advantage box plots had is that they can be drawn by hand. Now that computers are ubiquitous this is no longer valuable. Violin plots and bee swarm plots are better. Jittered strip plots can be okay if you're careful to avoid saturation (or more points added in the saturated region will disappear as they can't make it any darker).

I'll take a plain histogram/kde plot every day of the week over those damn violin plots. I think box plots are quite usefull as they are easy to read but only if you trust the author has actually looked at the histogram. And you can typically not trust the author to have done that.

Violin plots essentially are KDE plots, but you can put multiple of them on the same axes to compare groups.

Re: I've stopped using box plots (2021)

#70

Post by JumpCrisscross »

quenix wrote: post #43
SillyUsername wrote: post #8

So the diagram should not be used because of an education problem with some audiences? Isn't that a bit like banning cars because some people can't drive? Some diagrams are simply not for mass consumption and this is one, particularly because it is designed to illustrate an interpretation of ranges instead of the direct/linear representation of the raw data. Of course I'd illustrate this fact as a Venn diagram compar…

Driving isn't a medium of communication, so this is an apples to oranges comparison. If a medium of communication is misunderstood and found to be misleading to your audience, it doesn't really matter whether it's an education problem or not. It ceases to be a good communication medium. The entire purpose of data viz as the author discussed is to convey ideas to other people. The author argues that people tend to mis…

> Driving isn't a medium of communication

There is a lot of implicit (e.g. traffic signals) and explicit (e.g. indicators and horns) inter-driver communication that is at the heart of most crashes.

Post reply on HN