Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unlearning descriptive statistics

debrouwere.org

Post reply on HN
Print view View original

51–60 of 95 posts

Re: Unlearning descriptive statistics

#51

Post by somestag »

stdbrouw wrote: post #48
somestag wrote: post #15

I think the message of the article is great: move beyond the "standard" descriptions and pay more attention to what you're trying to show and who your audience is. That said, it's a slight pet peeve of mine when people recommend the median over the mean to describe center. The median, on its own, does not describe what is "typical" any more than the mean does; it just has a small advantage in that it will always map…

> In general, the median conveys remarkably little information. It's great at showing the center, but it gives zero indication of anything else. The median also has zero ability to make me coffee in the mornings, but I don't think I can hold that against it. I can imagine situations where getting people to track or listen to even a single number is tough and using the mean as a measure of both central tendency and th…

Clearly the best approach is to directly answer the question at hand with the most relevant numbers/statistics available. I'm all for that. I'd argue you'd still want to use the mean more than the median, but it's not really important at that point because you're painting the most complete picture you can.

There are times that, for whatever reason, someone is only presenting one statistic. News headlines are a big one. In this case, you clearly want to pick the "best" statistic available. There are people who think that the median is categorically better than the mean and dispense advice as such. (You can find them saying things like "the mean is worthless, the median would be much better" in the comments of discussion boards.) That's not what this article did, but the author did imply (in its title and language, if nothing else) that the median is better than the mean without offering any sort of weighing mechanism for which to choose. That's what I'm responding to, because it perpetuates the median > mean myth that's prevalent among certain groups of people.

Re: Unlearning descriptive statistics

#52

Post by somestag »

alexilliamson wrote: post #24
somestag wrote: post #15

I think the message of the article is great: move beyond the "standard" descriptions and pay more attention to what you're trying to show and who your audience is. That said, it's a slight pet peeve of mine when people recommend the median over the mean to describe center. The median, on its own, does not describe what is "typical" any more than the mean does; it just has a small advantage in that it will always map…

But I think the author was careful to sprinkle caveats so as to avoid universal recommendations. Rather, it take on a few common abuses. About the mean vs. median, it might be true that "mean" is appropriate in just as many contexts as "median", but at least in my opinion, people cite a "mean" when a "median" would have been appropriate more frequently than the reverse. There are times when neither are appropriate, b…

I agree with the article as a whole. I just personally believe that the median is generally worse to use than the mean, so I believe that if you're going to recommend the median over the mean, you should provide guidelines on when to do so. I know not everyone agrees with me on this (as evidenced by some responses in these comments) but that's the perspective I'm coming from.

> it might be true that "mean" is appropriate in just as many contexts as "median", but at least in my opinion, people cite a "mean" when a "median" would have been appropriate more frequently than the reverse.

Sure, but this is only true because the mean is used so much more. If the whole world was instead taught in grade school to use nothing but the median, there would be just as many (I'd argue more) misuses as there are now, except this time the median would be the offender. People using the median are more likely to be using it inappropriately because it's the nonstandard option, but that's only because they're not using it blindly. If people starting using the median just because, it would suffer the same issue.

Re: Unlearning descriptive statistics

#53

Post by wodenokoto »

Great article overall, but I wish it went a little deeper in explaining how we should interpret the statistics it recommend.

It basically says "don't use the normal numbers, use these instead, because they are closer to what you would expect them to mean"

However I don't find all of these statistics super obvious.

Re: Unlearning descriptive statistics

#54

Post by maxerickson »

closed wrote: post #19
somestag wrote: post #15

I think the message of the article is great: move beyond the "standard" descriptions and pay more attention to what you're trying to show and who your audience is. That said, it's a slight pet peeve of mine when people recommend the median over the mean to describe center. The median, on its own, does not describe what is "typical" any more than the mean does; it just has a small advantage in that it will always map…

Great points. Discrete data with few categories are a good case where the median might not cut it. As a contrived example.. data: 0 0 0 1 1 mean: 2 / 5 median: 0

With a small number of categories a full summary of the data easily fits into text. It might still be desirable to report a summary statistic, but omitting the totals for 2 or 3 categories and reporting a summary statistic is pretty sloppy.

Re: Unlearning descriptive statistics

#55

Post by em500 »

To answer the author's postscript: "why did nobody tell me this?"

Because you didn't pay attention at school? Means, medians, modes and percentiles were standard fare in my high school, and again in freshman university courses. And I didn't go to particularly expensive schools or elite universities. Then again, I also taught intro stat courses to BA students for a few years, and it was easy to tell that the vast majority of them cared very little, since understanding their uses wasn't exam material.

Re: Unlearning descriptive statistics

#56

Post by lenzm »

srean wrote: post #31
somestag wrote: post #15

I think the message of the article is great: move beyond the "standard" descriptions and pay more attention to what you're trying to show and who your audience is. That said, it's a slight pet peeve of mine when people recommend the median over the mean to describe center. The median, on its own, does not describe what is "typical" any more than the mean does; it just has a small advantage in that it will always map…

> when people recommend the median over the mean to describe center. The median, on its own, does not describe what is "typical" any more than the mean does; it just has a small advantage I think this swings the other extreme in selling the 'median' short. As long as we agree that it is only strictly meaningful to talk about the 'center' for symmetric distributions, median does a fine job. In fact in many realistic s…

It's a better than 50% chance - if the sample size is odd, you always get a real observation; if the sample size is even, you may still get a real observation (if the two median observations are equal).

Re: Unlearning descriptive statistics

#57

Post by keithpeter »

OliverJones wrote: post #12

This is terrific. In the field of web ops, I've had great success tracking the 95th percentile of request time rather than the mean, median, mode, or any other descriptive metric. The systems I worked with were, like many systems, ordinarily very efficient. That meant the mean and median metrics washed away the occasional troublesome request and hid it from the metrics dashboard. But knowing the 95th percentile was o…

Sounds fine to me: fits in with 'management by exception' - look for the extreme cases.

Standard in the industrial quality control world (2*sd above or below).

Re: Unlearning descriptive statistics

#59

Post by autokad »

somestag wrote: post #50
srean wrote: post #31

Earlier quoted context omitted.

> when people recommend the median over the mean to describe center. The median, on its own, does not describe what is "typical" any more than the mean does; it just has a small advantage I think this swings the other extreme in selling the 'median' short. As long as we agree that it is only strictly meaningful to talk about the 'center' for symmetric distributions, median does a fine job. In fact in many realistic s…

The median is worse than the mean in a skewed distribution if you want to take the skew into account. In fact one of the strengths of the mean is that it is sensitive to changes in the entire sample, rather than only part of it. To repeat my previous example, if the lowest 20% of household incomes drop because of changes to the tax code, that's something I'd generally want reflected in my "household income" statistic…

i agree with the article in that: if you want to take the screw into account, you look at the histogram. as a single number, I dont think the mean tells you anything about the skew/tail of the data any better than the median.

Re: Unlearning descriptive statistics

#60

Post by baq »

apathy wrote: post #16
shmageggy wrote: post #9

> But why do you want a number at all? Because lots of data analysis questions hinge upon the association between two data sets, and it's nice (crucial) to be able to quantify this value. Especially because > While statisticians are generally quite good at estimating a correlation from a picture and vice versa, most people are not. The author says > Still not happy and absolutely want a number? You would do well to s…

1) it is pretty amazing that normit transformations (map the quantiles of a non-normal distribution onto a Gaussian and use that) don't seem to be on this guy's radar. We use distributions with linearly additive and affine invariant properties (normal plus normal is normal, bernoulli plus bernoulli is bitwise bernoulli) because we find linear algebra very useful. Nonparametric tests and procedures erode your power; n…

i googled "normit transformations". the only non-publication link on the first page was (i can only assume) an automatic japanese translation...
Post reply on HN