121–130 of 173 posts
Looking particularly at latency measurements, I found the "How NOT to Measure Latency" [1] talk very illuminating. It goes quite deep into discussing how percentiles can be used and abused for measurement. [1]: https://www.infoq.com/presentations/latency-response-time/
Earlier quoted context omitted.
I watch this video once a year and send it to my co-workers whenever averages or medians shows up in a graph for public consumption.
Are the points written in a readable format anywhere?
Earlier quoted context omitted.
I heavily caution against the feeling that "standard deviation is a simple way to essentially include percentiles." The usefulness of the standard deviation depends on the distributions that you are working with. Heavy tailed distributions appear a fair amount in practice, and the combo of summary statistics mentioned would not do well on those. Also, Madars' comment in this thread is a beautiful example of this: 4 c…
I assume most of the distributions a marketing department would be dealing with are generally normal in which case stddev is a great way to analyze the data. This can be easily verified by just plotting said data and making sure the tails don't look weird.
I really enjoyed this post! The author also wrote an interactive demonstration of the concepts (using Desmos). It's super helpful. https://www.desmos.com/calculator/ty3jt8ftgs
Glad you liked it! I was so excited to actually be able to get to use Desmos for something work wise, I've been wanting to do it for years!
Earlier quoted context omitted.
i think of it as: if the data is gaussian, use a mean, otherwise go non-parametric (medians/percentiles). or put another way, if you can't model it, you're going to have to sort, or estimate a sort, because that's all that's really left to do. this shows up in things from estimating centers with means/percentiles to doing statistical tests with things like the wilcoxon tests.
Assume up front none of your measured latencies from a software networked system will be Gaussian, or you will die a painful death . Even ping times over the internet have no mean. The only good thing about means is you can combine them easily, but since they are probably a mathematical fiction, combining them is even worse. Use T-Digest or one of the other algorithms being highlighted here.
Earlier quoted context omitted.
Barchart is basically your percentiles (just more of them) so why not show it? Bars and whiskers could be more complicated for them but still the same sort of data
Barcharts across categorical data :P That is, the first bar is "Our Number" and the second bar is "Competitor's number."
Awhile ago I wrote a Python library called LiveStats[1] that computed any percentile for any amount of data using a fixed amount of memory per percentile. It uses an algorithm I found in an old paper[2] called P^2. It uses a polynomial to find good approximations. The reason I made this was an old Amazon interview question. The question was basically, "Find the median of a huge data set without sorting it," and the "…
> Find the median ... randomly evict items So, not find, but approximate. That's a different thing.
(that you sort yourself)
Awhile ago I wrote a Python library called LiveStats[1] that computed any percentile for any amount of data using a fixed amount of memory per percentile. It uses an algorithm I found in an old paper[2] called P^2. It uses a polynomial to find good approximations. The reason I made this was an old Amazon interview question. The question was basically, "Find the median of a huge data set without sorting it," and the "…
> The question was basically, "Find the median of a huge data set without sorting it," Isn't this done using a min heap and a max heap in conjuction?