Statistical Formulas For Programmers
evanmiller.org
Statistical Formulas For Programmers
1–10 of 66 posts
Re: Statistical Formulas For Programmers
#2Uhhhh, Evan Miller here. Not sure why my name is in the submitted title, but whatever.
The current selection on that page is somewhat limited, but I hope to grow it over time. The stuff at the beginning is pretty basic (e.g. standard deviation), but things get pretty gnarly by the time you get to the Kiefer equation. At some point I'll add some more references on how to implement things, e.g. find successive zeros of Bessel equations. For now it should be a good jumping-off point. Enjoy!
Re: Statistical Formulas For Programmers
#3Uhhhh, Evan Miller here. Not sure why my name is in the submitted title, but whatever. The current selection on that page is somewhat limited, but I hope to grow it over time. The stuff at the beginning is pretty basic (e.g. standard deviation), but things get pretty gnarly by the time you get to the Kiefer equation. At some point I'll add some more references on how to implement things, e.g. find successive zeros of…
Well Evan Miller, I thank you for this anyway. I look forward to checking it out more soon.
I have been thinking about collecting some stories about how people have put their knowledge of stats when programming (performance testing, user patterns examination, etc), what do you think? It would be a kind of applied stats for programmers thing I guess.
Re: Statistical Formulas For Programmers
#4The first example, "unbiased standard deviation" is mislabeled. The estimator of the variance is unbiased but the square root of an unbiased estimator is not itself unbiased. So it's not as nit-picky as it looks; it's either a brain fart or a hole in understanding (especially since the linked Wikipedia page discusses this issue)[1,2]
Not to be a dick, but getting the first example wrong like that doesn't inspire confidence in the rest of the post.
[1] https://en.wikipedia.org/wiki/Standard_deviation
[2] https://en.wikipedia.org/wiki/Unbiased_estimation_of_standar...
Re: Statistical Formulas For Programmers
#5Great start on an important topic. Quick extra info - For drawing a trend line it is often useful to have the intercept as well. Using y=mx+b line notation the best fit intercept is: \hat{b} = \bar{y} - m * \bar{x} [1]
Be great to see some pictures to illustrate the formulas and some mention of robust statistics as I find outliers to be a huge issue in application of statistical techniques.
Re: Statistical Formulas For Programmers
#6Hey Evan, from one statistics guy to another, thanks for fighting the good fight :).
The formulas might benefit from examples, especially with some of the more complicated cases (KS test and onwards). The important part of statistics comes from knowing _when_ to apply something, rather than _how_ to (that part is just math/numerical analysis).
A mention of the assumptions of each of these intervals would be good, too. Too often I see conclusions invalidated by using a probability model that doesn't make sense. This is a common failure I see with using a Wald interval for the slope of the regression line.
Re: Statistical Formulas For Programmers
#7The first example, "unbiased standard deviation" is mislabeled. The estimator of the variance is unbiased but the square root of an unbiased estimator is not itself unbiased. So it's not as nit-picky as it looks; it's either a brain fart or a hole in understanding (especially since the linked Wikipedia page discusses this issue)[1,2] Not to be a dick, but getting the first example wrong like that doesn't inspire conf…
Thanks! I have relabeled it. (In a previous draft the entry was for unbiased variance and the inaccuracy slipped in during the transition.)
Re: Statistical Formulas For Programmers
#8Hey Evan, from one statistics guy to another, thanks for fighting the good fight :). The formulas might benefit from examples, especially with some of the more complicated cases (KS test and onwards). The important part of statistics comes from knowing _when_ to apply something, rather than _how_ to (that part is just math/numerical analysis). A mention of the assumptions of each of these intervals would be good, too…
Agreed 100%! As the introduction says this is supposed to be a "cheat-sheet" and a jumping-off point for further discovery... I'd love to see (and write!) more posts on when to apply things.
Re: Statistical Formulas For Programmers
#9The first example, "unbiased standard deviation" is mislabeled. The estimator of the variance is unbiased but the square root of an unbiased estimator is not itself unbiased. So it's not as nit-picky as it looks; it's either a brain fart or a hole in understanding (especially since the linked Wikipedia page discusses this issue)[1,2] Not to be a dick, but getting the first example wrong like that doesn't inspire conf…
Seems more like a brain fart than an error in understanding, especially because he didn't highlight why it was important to have unbiasedness in the first place. Having to explain the unbiased estimator for standard deviation would probably not have contributed much. It's more of a technicality than anything of philosophical importance--the square root of the unbiased estimator of sample variance is pretty accurate.
The usage of n-1 instead of n in the denominator was a question I was always asked when I TA'ed for intro statistics classes in grad school. An explanation of unbiasedness might be warranted if this is to be an introductory primer.
Re: Statistical Formulas For Programmers
#10Hey Evan, from one statistics guy to another, thanks for fighting the good fight :). The formulas might benefit from examples, especially with some of the more complicated cases (KS test and onwards). The important part of statistics comes from knowing _when_ to apply something, rather than _how_ to (that part is just math/numerical analysis). A mention of the assumptions of each of these intervals would be good, too…
While you guys are here, can you recommend a good intro book for statistics?