Why We Need a Statistical Revolution • Hacker News Board

Re: Why We Need a Statistical Revolution

#11

Post by pXMzR2A » Sun, May 17, 2015, 9:55 PM UTC

Step 1: Establish disciplines that ridicule radical criticism.

Step 2: Hire undergraduate math/stats students who took criticism courses at gun point and hated them.

Step 3: ...

Step 4: "OMG Ponies and revolution"

Sir, you do not need a revolution, because it has already occured. You simply need employees who have read critical approaches to statistics AS AN INTEGRAL PART of their stats education. Those people are usually in

- feminist studies (did the requested stats revolution)

- gender studies (ibid, different name)

- women's studies (ibid, different name)

- lgbt studies (exploded feminist studies ftw)

- african american studies (ibid)

- critical studies (sociology, know of mainstream criticism only)

- american studies (good with convoluted language)

Re: Why We Need a Statistical Revolution

#12

Post by rcthompson » Sun, May 17, 2015, 10:42 PM UTC

the8472 wrote: post #7

I always found the initial assumption that things are normally distributed quite odd. For example if you work with latencies then the normal distribution is a terrible thing to use. The closer to zero you get the more skewed it has to be, simply because everything that can influence the result will mostly change it in one direction. So in that particular case an Erlang distribution might be more applicable. And even…

The assumption is reasonable for a lot of cases because of the central limit theorem. In particular, it applies if the variable of interest is the sum of a large number of independent random variables. For example, the number of heads in N coin flips is the sum of the number of heads in each of N independent experiments each consisting of a single flip.

Of course, there are also a lot of important cases where that N isn't quite large enough to justify the normal distribution, or where the quantity of interest is not a sum of a large number of small effects, so the CLT doesn't apply at all.

Re: Why We Need a Statistical Revolution

#13

Post by oakwood » Sun, May 17, 2015, 11:56 PM UTC

Terribly written article. This guys lives in an academic cave.

Re: Why We Need a Statistical Revolution

#14

Post by jkyle » Mon, May 18, 2015, 2:35 AM UTC

The scenario presented by the author seems more explicit and nefarious than the typical "p-hunting" that I've seen in academia. Though I've seen the situation he presents, successively trying different models or parameters until you find the expected effect, it's pretty obviously unethical.

What I've seen much, much more of goes something like this. Say your lab does research that requires a very large quantity of data that's very expensive to acquire. In academia, labor is relatively cheap and funding can be difficult to come by. So a dataset that consumed potentially 100's of thousands of funding dollars represents a significant resource investment.

So what do you do when the study is done? You try to figure out every way possible to reuse that data. And the most common way of doing that is running it using previous models looking for effects...enough people keep looking long enough and, voila!, you find a significance.

In the end, it's the same bias. But the motivation is different and when I've pointed out this is "p-hunting" most just give a blank stare.

Re: Why We Need a Statistical Revolution

#15

Post by glesica » Mon, May 18, 2015, 2:55 PM UTC

ende wrote: post #6

glesica wrote: post #2

The dangers of fishing for p values (to focus on those paragraphs) are covered pretty extensively in undergraduate courses in, at the very least, economics and political science. Something happens between that point, and the point at which a researcher is trying to publish in a journal. I think it is fairly obvious that the problem is related to competition for funding and advancement. Of course this is probably a mu…

There's another potential source of the problem: the time gap in between those undergraduate lessons are learned in the abstract and the later point in time when the exercise if those lessons is required. Unexercised knowledge atrophies like any other muscle. Perhaps rather than placing so much emphasis on memorization recall exams, undergraduate evaluation should be based on exercising the research and peer review p…

That's what we did in undergraduate. Practically every class I had required a research paper. We didn't always have to use statistics, but we were graded on methodology. Recall exams were rare, most exams involved interpretation and explanation. Of course I studied political science, things may be different in other fields.