I've stopped using box plots (2021) • Hacker News Board

Re: I've stopped using box plots (2021)

#81

Post by scrollaway » Sun, Jun 23, 2024, 9:43 AM UTC

SillyUsername wrote: post #53

scrollaway wrote: post #49

Earlier quoted context omitted.

Yeah I'm shocked at the awful quality of comments here. This is a clear and straightforward article laying out the issues with box plots and appropriate alternatives, from a professional who works in the field and spends his life explaining these. And still half the comments are like "But I know better!"... yeah, I'd wager most here don't.

I'm qualified in maths related computing and statistics to exam invigilator level, if that helps offset your bias.

No bias -- By commenting a lot, you're overrepresenting the average HN audience. Which kind of nullifies your point, doesn't it?

You argue in other comments that it's just an education problem, but box plots are used with people who don't have this exact education you mention, and the article explains that a drawback of box plots is exactly that it isn't intuitive and takes several minutes of explanations.

In other words, the article says "I've stopped using this because they require education", and your retort is "Don't stop using these, you just need to educate people".

Re: I've stopped using box plots (2021)

#82

Post by cqqxo4zV46cp » Sun, Jun 23, 2024, 9:44 AM UTC

SillyUsername wrote: post #59

quenix wrote: post #43

Earlier quoted context omitted.

Driving isn't a medium of communication, so this is an apples to oranges comparison. If a medium of communication is misunderstood and found to be misleading to your audience, it doesn't really matter whether it's an education problem or not. It ceases to be a good communication medium. The entire purpose of data viz as the author discussed is to convey ideas to other people. The author argues that people tend to mis…

As I've mentioned in other comments less succinctly, data hiding is sometimes useful of for drawing attention to other areas. There are the better graphs the author mentioned for general purpose use, but the graph itself isn't at fault any more than using a bar chart with a poor scale (e.g omit 0-20) to do the same hiding.

What specific issue do you have with this article? “The graph itself isn’t at fault” is very “guns don’t kill people, people kill people”. Who cares? This distinction is utterly meaningless semantics. Why do you feel a need to ‘stand up’ for box plots? Why is this a tribalistic religious war?

Re: I've stopped using box plots (2021)

#83

Post by Evidlo » Sun, Jun 23, 2024, 9:44 AM UTC

sigmoid10 wrote: post #32

Beldin wrote: post #24

Earlier quoted context omitted.

> Box plots [...] assume that your data follows a bell/gaussian shape. Not sure how to square that with this statement on Wikipedia's page on box plots: Box plots are non-parametric: they display variation in samples of a statistical population without making any assumptions of the underlying statistical distribution[3]

If you want to see why that is not fully correct you should read the article. For a box plot you need to calculate mean, variance and certain percentiles. These values don't make sense if your distribution does not follow a certain shape (because these values unambiguously define such a shape). See the examples in the article for what happens if you still try to use them in those cases. You can still extract the valu…

> So you can no longer reverse the process

I've never understood this to be the purpose of a boxplot, only a means of visualizing a distribution's quartiles.

You've gotten a flood of comments from upset people, so I'll keep it short by saying that a boxplot doesn't actually do what you claim for Gaussians, as the 0 and 100 percentile "whiskers" would be at plus/minus infinity. As for a bounded bell-shaped distribution, there are several non-unique ways to define such a distribution.

Re: I've stopped using box plots (2021)

#84

Post by cqqxo4zV46cp » Sun, Jun 23, 2024, 9:50 AM UTC

SillyUsername wrote: post #28

317070 wrote: post #17

Earlier quoted context omitted.

It's not like banning cars, it is like banning horse carriages on high ways. We have better technology nowadays, including for plotting, so why not ditch the old? The author of the blog post has some good arguments. From your post, I cannot distill an argument as to why you would prefer specifically a box plot over a strip plot.

Yes the other diagrams are better for mass consumption, and illustrating direct representation of the data distribution. But that's not the purpose of a box diagram and the article even did a side by side comparison showing an apples and oranges comparison of 2 total different representations of the data. Those diagrams were never meant to represent the data in the same way. The article simply could have shown a bett…

IIn all of your replies you make snide reference to “general audiences”, “mass consumption”, etc. You very obviously place yourself in a higher class because of your ability to correctly interpret box plots. Can we please just move past that though? The vast vast vast majority of box plots are for “general consumption”. The vast vast majority of box plots are used in place of a more suitable chart type. You seem to be arguing that, because a box plot is hypothetically suitable for some (in the grand scheme of things) corner case, that the author’s point is faulty. I think that you are completely overstating the importance of the hypothetical ‘correct case’. You’re getting stuck on a point that nobody, least of all the author, is making.

Re: I've stopped using box plots (2021)

#85

Post by scrollaway » Sun, Jun 23, 2024, 9:50 AM UTC

cjk2 wrote: post #75

ohmyiv wrote: post #65

Earlier quoted context omitted.

> No you shouldn’t stop using box plots. You should use them for when they are appropriate Yes, the author is aware of that. They even stated so: > Despite making more visual sense than box plots, I still wouldn’t recommend these design concepts or box plots in most situations because… Seems a few people missed the "in most situations" part. He's saying he stopped using them for whatever reasons because it isn't work…

The author doesn't use the correct terminology and does not understand box plots themselves so they are in no position to explain them to anyone. They explain in terms of absolutes with no rational or scientific explanation and entirely miss the point of the methodology and tools. That is a not a good position to start or a good person to take advice from. Not only that, the cases presented are likely better dealt wi…

Is this sarcasm?

I'm not one to appeal to authority but "author should take a course" is akin to ad hominem when a quick look at their profile (https://www.practicalreporting.com/about-nick-desbarats - https://www.linkedin.com/in/nickdesbarats/) tells you that he's been doing dataviz and statistics for a long time.

Re: I've stopped using box plots (2021)

#86

Post by singingfish » Sun, Jun 23, 2024, 9:50 AM UTC

And no mention of notched box plots which make a lot of the troublesome aspects go away?

Re: I've stopped using box plots (2021)

#87

Post by kkfx » Sun, Jun 23, 2024, 9:54 AM UTC

Honestly? I do not care much about charts in general, while I do care much about the availability of the data used to produce a chart... In way too much cases I see plots and no data, sometimes data are there but not easy to use, and another thing I do care is the ability to tweak a graph.

The above are between the reasons I prefer remote meeting where data are to be shown instead of in person: anyone attending should have a computer ready to use and IF data are shared and ready usable I can live tweaks a plot ad reason on it while I listen end eventually pose relevant questions shown at my own turn something. Surely not all presentations are meant to be interactive session, but being able to interact even in async form reading a journal article, playing with the data and eventually drop a mail to the author is a nice thing, typically uselessly hard today where in tech term it can be extremely simple.

That's another reason I have presentation software/office automation one instead of plain org-mode, Jupyter, R Studio etc because change things it's hard while it should be easy. Org-mode is excellent to present but not really interactive, I have to regenerate plots to see changes or push data to external software, Jupyter is not really meant to present, R Studio offer nice LaTeX integration and tabular view but do not offer nice means to present, though they are still FAR better then presentation software and even if have some safety aspects to be taken into account I prefer countless of time receiving an active document (org-mode, jupyter notebook etc) instead of a pdf or even worse some office formats.

Re: I've stopped using box plots (2021)

#88

Post by quietbritishjim » Sun, Jun 23, 2024, 9:58 AM UTC

rcxdude wrote: post #40

sigmoid10 wrote: post #12

>box plots always make distributions look bell shaped I feel like this is where the confusion stems from for the author and everyone else here. Box plots don't make anything bell shaped (they don't change the distribution), they assume that your data follows a bell/gaussian shape. This is correct in cases where the central limit theorem can be applied (which is almost everywhere) - but when that is not the case, the…

Nah, there's nothing in a box plot which assumes a bell-shape. It does, however just visualise the parameters which reasonably well characterize a smooth single-mode distribution regardless of the underlying distribution. So it's a valid criticism of using box plots, especially when the alternatives can just as well visualise a bell-shaped distribution, as well as showing when it is not.

> smooth single-mode distribution

That IS a bell curve. While it's true that the Guassian distribution is often called a bell curve or even "the" bell curve, a non-Guassian single mode distribution is still absolutely bell shaped in a general sense.

So, although you started your comment with "nah", you're actually in agreement with the content you replied to.

Re: I've stopped using box plots (2021)

#89

Post by ohmyiv » Sun, Jun 23, 2024, 10:01 AM UTC

cjk2 wrote: post #75

ohmyiv wrote: post #65

Earlier quoted context omitted.

> No you shouldn’t stop using box plots. You should use them for when they are appropriate Yes, the author is aware of that. They even stated so: > Despite making more visual sense than box plots, I still wouldn’t recommend these design concepts or box plots in most situations because… Seems a few people missed the "in most situations" part. He's saying he stopped using them for whatever reasons because it isn't work…

The author doesn't use the correct terminology and does not understand box plots themselves so they are in no position to explain them to anyone. They explain in terms of absolutes with no rational or scientific explanation and entirely miss the point of the methodology and tools. That is a not a good position to start or a good person to take advice from. Not only that, the cases presented are likely better dealt wi…

> The author is way out of their depth and should retract the article and take a formal, accredited statistics course.

Maybe you should learn about the author before you make such assumptions. I find it hilarious you think he should take statistics courses when he teaches data visualization workshops to places like NASA, IRS, and the UN.

I'm done with this thread. Such a joke.

Re: I've stopped using box plots (2021)

#90

Post by karmakaze » Sun, Jun 23, 2024, 10:14 AM UTC

> There are other distribution chart types that can be useful in specific situations, such as frequency polygons, violin plots, cumulative distribution plots, and bee swarm plots, but the three types that I described above are the easiest ones to grasp, and are able to communicate most of the insights that are needed for day-to-day decision-making in most organizations. (I’m not mentioning histograms here because they’re generally only useful for visualizing a single set of values, whereas box plots and their alternatives are for visualizing multiple sets of values, which is a different use case.)

There's generalizations and 'specific situations' which the author considers worthy of some plots, and other specific situations that the author doesn't consider worthy of other plots. At best, don't use box plots if your distributions do not have a single mode and may likely be misinterpreted is my takeaway. Here's a rant against violin plots by my fave physicist ranter[0] (not Sabine), so maybe never use them.

[0] https://youtu.be/_0QMKFzW9fw?si=4VM4DT9Q1zEnV93A