Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How percentile approximation works and why it's more useful than averages

blog.timescale.com

Post reply on HN
Print view View original

31–40 of 173 posts

Re: How percentile approximation works and why it's more useful than averages

#31

Post by djk447 »

esyir wrote: post #26
waynecochran wrote: post #12

Earlier quoted context omitted.

Yeah -- std deviation has a similar problem to the mean in that it doesn't give you a full picture unless the distribution is close to normal / gaussian.

Pretty much why summary statistics often give the IQR, which gives some idea to the skew and shape of the distribution as well. Unfortunately, BD and marketing just want a single number to show that the value is bigger and hate anything more complicated than a barchart.

NB: Post author here.

We've been meaning to add IQR as an accessor function for these, may have to go back and do it...the frequency trails [1] stuff from Brendan Gregg also goes into some of this and it's really cool as a visualization.

[1]: https://www.brendangregg.com/FrequencyTrails/mean.html

Re: How percentile approximation works and why it's more useful than averages

#32

Post by s_gourichon »

pdpi wrote: post #24
madars wrote: post #8

Good opportunity to plug https://en.wikipedia.org/wiki/Anscombe%27s_quartet : if you don't know much about the underlying distribution, simple statistics don't describe it well. From Wikipedia description: Anscombe's quartet comprises four data sets that have nearly identical simple descriptive statistics, yet have very different distributions and appear very different when graphed. Each dataset consists of eleven (x…

There's a fun paper by Autodesk where they make datasets that look whatever way you want them to. https://www.autodesk.com/research/publications/same-stats-di...

Yes, a both funny and insightful lesson on how weak basic indicators (mean, standard deviation, correlation) can be, the interest and limits of box-plot with quartiles and whiskers, the benefit of the violin-plot.

Definitely worth a quick look.

Re: How percentile approximation works and why it's more useful than averages

#33

Post by axpy906 »

There’s something called a five number summary in statistics: mean, median, standard deviation, 25th percentile and 75th percentile.

The bonus is that the 75th - 50th gives you the interquartile range.

Mean is not a robust measure and as such you need to look at variety to truly understand the spread of your data.

Re: How percentile approximation works and why it's more useful than averages

#34

Post by bluesmoon »

axpy906 wrote: post #33

There’s something called a five number summary in statistics: mean, median, standard deviation, 25th percentile and 75th percentile. The bonus is that the 75th - 50th gives you the interquartile range. Mean is not a robust measure and as such you need to look at variety to truly understand the spread of your data.

IQR is 75th - 25th, aka, the middle-50%

Re: How percentile approximation works and why it's more useful than averages

#35

Post by pdpi »

djk447 wrote: post #25
pdpi wrote: post #24

Earlier quoted context omitted.

There's a fun paper by Autodesk where they make datasets that look whatever way you want them to. https://www.autodesk.com/research/publications/same-stats-di...

NB: Post author here. This is great! So fun...will have to use in the future...

While I have your attention...

> For the median and average to be equal, the points less than the median and greater than the median must have the same distribution (i.e., there must be the same number of points that are somewhat larger and somewhat smaller and much larger and much smaller).

[0, 2, 5, 9, 9] has both median and mean = 5, but the two sides don't really have the same distribution.

Re: How percentile approximation works and why it's more useful than averages

#36

Post by djk447 »

pdpi wrote: post #35
djk447 wrote: post #25

Earlier quoted context omitted.

NB: Post author here. This is great! So fun...will have to use in the future...

While I have your attention... > For the median and average to be equal, the points less than the median and greater than the median must have the same distribution (i.e., there must be the same number of points that are somewhat larger and somewhat smaller and much larger and much smaller). [0, 2, 5, 9, 9] has both median and mean = 5, but the two sides don't really have the same distribution.

Totally true...thoughts on how I could rephrase? I guess it's more the "weight" of points greater than and less than the median should be the same, so symmetric distributions definitely have it, asymmetric may or may not. Definitely open to revising...

Re: How percentile approximation works and why it's more useful than averages

#38

Post by hnuser123456 »

Gamers have an intuitive sense of this. Your average framerate can be arbitrarily high, but if you have a big stutter every second between the smooth moments, then a lower but more consistent framerate may be preferable, typically expressed as the 1% and 0.1% slowest frames, which at a relatively typical 100fps, represents the slowest frame every second and every 10 seconds.

Re: How percentile approximation works and why it's more useful than averages

#39

Post by pdpi »

djk447 wrote: post #36
pdpi wrote: post #35

Earlier quoted context omitted.

While I have your attention... > For the median and average to be equal, the points less than the median and greater than the median must have the same distribution (i.e., there must be the same number of points that are somewhat larger and somewhat smaller and much larger and much smaller). [0, 2, 5, 9, 9] has both median and mean = 5, but the two sides don't really have the same distribution.

Totally true...thoughts on how I could rephrase? I guess it's more the "weight" of points greater than and less than the median should be the same, so symmetric distributions definitely have it, asymmetric may or may not. Definitely open to revising...

> symmetric distributions definitely have it, asymmetric may or may not

Doesn't have to be any more complicated than that. It's more a curio than an important point anyway :)

Re: How percentile approximation works and why it's more useful than averages

#40

Post by vvern »

djk447 wrote: post #23
cyral wrote: post #13

Earlier quoted context omitted.

There are some newer data structures that take this to the next level such as T-Digest[1], which remains extremely accurate even when determining percentiles at the very tail end (like 99.999%) [1]: https://arxiv.org/pdf/1902.04023.pdf / https://github.com/tdunning/t-digest

NB: Post author here. Yeah, that was one of the reasons we chose it as one of the ones to implement, seemed like that was a really interesting tradeoff, we also used uddsketch[1] which provides relative error guarantees, which is pretty nifty. We thought they provided different enough tradeoffs that we wanted to implement both. [1]: https://arxiv.org/abs/1908.10693

Is it using https://github.com/tvondra/tdigest under the hood, or a separate implementation?
Post reply on HN