Hey everyone, I'm the author of this guide. It's come full circle -- I posted it a week ago in a "what are you working on?" Ask HN post, someone posted it to Metafilter and reddit, and it made its way to Boing Boing and Daily Kos before coming back here. I'm currently working on expanding the guide to book length, and considering options for publication (self-publishing, commercial publishers, etc.). It seems like a…
Statistics Done Wrong – The woefully complete guide
21–30 of 71 posts
Re: Statistics Done Wrong – The woefully complete guide
#22If you are interested in the difference of a metric scaled quantity between two groups do the following:
1.) Add 4-5 plausible control variables that you do not document in advance (questionaire, sex, age...).
2.) Write a r-script that helps you do the following: Whenever you have tested a person increment your dataset with the persons result and run a:
t-test
u-test
ordinal logistic regression over some possible bucket combinations.
3.) Do this over all permutations of the control variables. Have the script ring a loud bell when significance is achieved so data collection is stopped immediately. An added bonus is that you will likely get a significant result with a small n which enables you to do a reversed power analysis.
Now you can report that your theoretical research implied a strong effect size so you choose an appropriate small n which, as expected, yielded a significant result ;)
Re: Statistics Done Wrong – The woefully complete guide
#23Earlier quoted context omitted.
Isn't that why the name "Data Science" was invented?
Data science sure sounds sexier...ish. The irony of people who use the "damned lies and statistics" quote snidely is that the "statistics" part is not referring to the field Statistics but the plural version of the noun statistics, which of course are easily abused. The field of Statistics is all about NOT abusing statistics.
Re: Statistics Done Wrong – The woefully complete guide
#24If I was a billionaire, I would set up some sort of screening lab for scientific/academic/research papers. There would be a statistics division for evaluating the application of statistical methods being used; a replication division for checking that experiments do actually replicate; and a corruption division for investigating suspicious influences on the research. It would be tempting to then generate some sort of…
Re: Statistics Done Wrong – The woefully complete guide
#25At this point, if someone published a study stating that we needed to eat not to die, I'd be skeptical of it.
Re: Statistics Done Wrong – The woefully complete guide
#26One way to do statistics less wrong is to move from statistical testing to statistical modelling. This is what we are trying to support with BayesHive at https://bayeshive.com Other ways of doing this include JAGS ( http://mcmc-jags.sourceforge.net/ ) and Stan ( http://mc-stan.org/ ) The advantage of statistical modelling is that it makes your assumptions very explicit, and there is more of an emphasis on effect size…
Re: Statistics Done Wrong – The woefully complete guide
#27If I was a billionaire, I would set up some sort of screening lab for scientific/academic/research papers. There would be a statistics division for evaluating the application of statistical methods being used; a replication division for checking that experiments do actually replicate; and a corruption division for investigating suspicious influences on the research. It would be tempting to then generate some sort of…
Something like that idea has actually already been the inspiration for at least one startup: MetaMed ( http://en.wikipedia.org/wiki/MetaMed , http://nymag.com/health/bestdoctors/2013/metamed-personalize... ) does meta-level analysis of the medical literature to determine which treatments seem effective for rare conditions, taking into account the sample size, statistical methodology, funding sources, etc. of each stu…
Re: Statistics Done Wrong – The woefully complete guide
#28Re: Statistics Done Wrong – The woefully complete guide
#29Hey everyone, I'm the author of this guide. It's come full circle -- I posted it a week ago in a "what are you working on?" Ask HN post, someone posted it to Metafilter and reddit, and it made its way to Boing Boing and Daily Kos before coming back here. I'm currently working on expanding the guide to book length, and considering options for publication (self-publishing, commercial publishers, etc.). It seems like a…
Is there any way to print the entire book as a document? Do you accept donations?
Sphinx can output Latex for printing to PDF. We'd just need the source.
BTW, excellent refresher for statistical screw-ups, I had forgotten a quarter of these (and never learned the rest.)
Re: Statistics Done Wrong – The woefully complete guide
#30One way to do statistics less wrong is to move from statistical testing to statistical modelling. This is what we are trying to support with BayesHive at https://bayeshive.com Other ways of doing this include JAGS ( http://mcmc-jags.sourceforge.net/ ) and Stan ( http://mc-stan.org/ ) The advantage of statistical modelling is that it makes your assumptions very explicit, and there is more of an emphasis on effect size…
BayesHive is very interesting! I couldn't find any details on pricing, though?