Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unlearning descriptive statistics

debrouwere.org

Post reply on HN
Print view View original

81–90 of 95 posts

Re: Unlearning descriptive statistics

#81

Post by telotortium »

OliverJones wrote: post #12

This is terrific. In the field of web ops, I've had great success tracking the 95th percentile of request time rather than the mean, median, mode, or any other descriptive metric. The systems I worked with were, like many systems, ordinarily very efficient. That meant the mean and median metrics washed away the occasional troublesome request and hid it from the metrics dashboard. But knowing the 95th percentile was o…

Gil Tene has an entire talk about this point called "How NOT to measure latency" (https://www.youtube.com/watch?v=lJ8ydIuPFeU). In many circumstances, a web service (either server or client) has large numbers of dependencies, so a user only has to hit 95th-percentile latency for one dependency to have their overall latency significantly hurt. In other words, users may encounter the "rare" bad case on a majority of requests. Thus, even higher percentiles should be tightly controlled in order to truly keep a web service's overall latency under control.

Re: Unlearning descriptive statistics

#82

Post by krick »

stdbrouw wrote: post #79
krick wrote: post #74

Can somebody recommend a good book (or any other resource) on statistics? I mean kind of stuff this post talks about: descriptives, tests, all the basic stuff. Despite being somewhat familiar with that, I sometimes feel I really lack the solid understanding of the subject and am longing for something explanatory, with real life examples and exercises.

Think Stats ( http://greenteapress.com/wp/think-stats-2e/ ) and the follow-up Think Bayes are awesome if you're a coder. https://bblais.github.io/statistical-inference-for-everyone-... is a complete rethink of how basic statistics should be taught and is excellent as well.

Thanks, looks awesome.

Re: Unlearning descriptive statistics

#83

Post by IndianAstronaut »

CalChris wrote: post #45
andy_wrote wrote: post #8

For readers who are OK with some math, I recommend John Myles White's eye-opening post about means, medians, and modes: http://www.johnmyleswhite.com/notebook/2013/03/22/modes-medi... He describes these summary descriptive stats in terms of what penalty function they minimize: mean minimizes L2, median minimizes L1, mode minimizes L0. A single-number statistic is _going_ to leave things out, so if you must boil thing…

Statistics are reductionist! Well yes, that is their purpose.

My statistics professor once told us that statistics are a shadow of the truth, not the actual truth itself.

Re: Unlearning descriptive statistics

#84

Post by srean »

somestag wrote: post #50
srean wrote: post #31

Earlier quoted context omitted.

> when people recommend the median over the mean to describe center. The median, on its own, does not describe what is "typical" any more than the mean does; it just has a small advantage I think this swings the other extreme in selling the 'median' short. As long as we agree that it is only strictly meaningful to talk about the 'center' for symmetric distributions, median does a fine job. In fact in many realistic s…

The median is worse than the mean in a skewed distribution if you want to take the skew into account. In fact one of the strengths of the mean is that it is sensitive to changes in the entire sample, rather than only part of it. To repeat my previous example, if the lowest 20% of household incomes drop because of changes to the tax code, that's something I'd generally want reflected in my "household income" statistic…

> When people say the median is "better" than the mean, they often can't explain exactly why (other than to cite skew, which I've addressed).

Well, 40~50 years of literature on robust statistics happens to disagree with the claim 'people' don't know what they mean (pun unintended) when the say the median is better. Furthermore skew has less to do with that argument than heavy tails. heavy tails are extremely (ok sorry, now its an insider pun) common.

Re: Unlearning descriptive statistics

#85

Post by srean »

apathy wrote: post #72
srean wrote: post #34

Earlier quoted context omitted.

I largely agree with what you are saying but estimating the population transformation that makes the transformed data Gaussian from a finite sample is far from trivial. If you have any pointers to results that show distribution free guarantee of increased power I would be super happy to read. Here's a question for you , why not just deal with the quantiles directly (for example with quantile regression for regression…

quantile regression is computationally intensive and inverse transformations usually less so. Although you could certainly make the case that, given enough data, quantile regression better captures what we actually want to find, most of the time (i.e. how's this effect diverge towards the extremes). Normit typically (not sure if universally) has the lovely property of giving you something like a marginal t-test witho…

Thanks for responding, we are definitely in agreement. I have used both, in my experience quantile regression seems better behaved at the tail than quantile transformation in regression tasks. I think if one can make the transform conditional on the covariate they would be comparable.

Re: Unlearning descriptive statistics

#86

Post by srean »

lenzm wrote: post #56
srean wrote: post #31

Earlier quoted context omitted.

> when people recommend the median over the mean to describe center. The median, on its own, does not describe what is "typical" any more than the mean does; it just has a small advantage I think this swings the other extreme in selling the 'median' short. As long as we agree that it is only strictly meaningful to talk about the 'center' for symmetric distributions, median does a fine job. In fact in many realistic s…

It's a better than 50% chance - if the sample size is odd, you always get a real observation; if the sample size is even, you may still get a real observation (if the two median observations are equal).

You are indeed correct, I was incorrectly assuming a continuous density, in which case the probability of a tie would have been zero.

Re: Unlearning descriptive statistics

#87

Post by kriro »

somestag wrote: post #15

I think the message of the article is great: move beyond the "standard" descriptions and pay more attention to what you're trying to show and who your audience is. That said, it's a slight pet peeve of mine when people recommend the median over the mean to describe center. The median, on its own, does not describe what is "typical" any more than the mean does; it just has a small advantage in that it will always map…

"""The median, on its own, does not describe what is "typical" any more than the mean does; it just has a small advantage in that it will always map a real observation"""

Only if you have an odd number of data points or the two middling data points have the same value. Example: I have four people with these sizes in cm: 120, 160, 180, 200...the median would be (160+180)/2=170 which is the size of none of the people.

Re: Unlearning descriptive statistics

#88

Post by kriro »

On a somewhat related note, I think it's interesting that there is data on how people perceive information but that isn't really taught in descriptive statistics (at least it wasn't at my university). I feel like data representation and charts/graphs should be more of a focus if the title of the class is descriptive statistics. I hadn't even heard about "How to Lie with Statistics" or "The Visual Display of Quantitative Information" before randomly finding them mentioned somewhere. I simply learned about some standard graphs without much explanation on why you use them or how they impact the understanding of the audience.

Re: Unlearning descriptive statistics

#89

Post by ttub »

johnmyleswhite wrote: post #13
andy_wrote wrote: post #8

For readers who are OK with some math, I recommend John Myles White's eye-opening post about means, medians, and modes: http://www.johnmyleswhite.com/notebook/2013/03/22/modes-medi... He describes these summary descriptive stats in terms of what penalty function they minimize: mean minimizes L2, median minimizes L1, mode minimizes L0. A single-number statistic is _going_ to leave things out, so if you must boil thing…

Glad you enjoyed that post so much. It really is a shame that we do such a bad job of teaching students about the inherent subjectivity of descriptive statistics and let students leave their courses with dangerous ideas about the existence of a Holy Grail statistic that will solve all of their problems.

The idea of misinterpreting metrics is a very general idea and is not specific to statistics.

Humans want to distill vast amounts of information to a more manageable amount, like for example a single number.

Equity analysts look at accounting metrics, psychologists look at psychometrics test cores, doctors look at some function of blood pressure, etc etc.

Any person with deductive and sceptical mental faculties in place, will recognize that these are all simplifications, and cannot be used to deliver a unified truth.

Also, being aware of this has very little to do with being technical or not (for example, plenty of programmers only look at a CPU's clock speed to gauge performance).

Anyhow, nice post.

Re: Unlearning descriptive statistics

#90

Post by [deleted] »

andy_wrote wrote: post #8

For readers who are OK with some math, I recommend John Myles White's eye-opening post about means, medians, and modes: http://www.johnmyleswhite.com/notebook/2013/03/22/modes-medi... He describes these summary descriptive stats in terms of what penalty function they minimize: mean minimizes L2, median minimizes L1, mode minimizes L0. A single-number statistic is _going_ to leave things out, so if you must boil thing…

[deleted]
Post reply on HN