Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We found only one-third of published psychology research is reliable – now what?

theconversation.com

Post reply on HN
Print view View original

21–30 of 74 posts

Re: We found only one-third of published psychology research is reliable – now what?

#21

Post by rjruxhchd »

Far more worrying is that only a third of economics findings can be replicated without the author's help, and yet these are the supposedly scientific findings our politicians insist on basing policy off of. Even if policy seems counterintuitive and clearly harmful to the general public, we're given the explanation "because economics says so." It's clear now from the Federal Reserve replicability study that we're being duped.

http://www.federalreserve.gov/econresdata/feds/2015/files/20...

Re: We found only one-third of published psychology research is reliable – now what?

#22

Post by geomark »

Let's be clear on the differencen between reproducible and replication. Reproducible is when an experiment's data and methods are published and other researchers can reproduce the same statistical results. An experiment that has been replicated is a new experiment that applies the same methods to a new set of subjects and arrives at statistically equivalent results.

The distinction is important. Reproducible research is a lot easier but there are still famous examples of attempts to reproduce published results that revealed serious flaws. Cases where the data are kept secret to thwart reproducibility are thoroughly suspect. Replication is a harder problem, both in being able to perform a potentially costly and difficult experiment, and in the statistical analysis to show equivalent results.

Re: We found only one-third of published psychology research is reliable – now what?

#23

Post by GalacticDomin8r »

subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

> Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony.

Yup, and now that the science in question was attempted to be verified and shown to be poor, we can improve the process. Try that with religion.

> Now that science is proven to be systemically corrupt

Huh? Anyone who took psychological research as gospel does so at their own risk and ignorance. We've long known psychology is a soft science little better than voodoo. For example, most of the psychological establishment still default to 12 step programs for addictions. Guess what? It works about as well as any faith based healing program.

This is just another measure of validating the scientific method. Which is to say we don't know everything but we're learning.

Re: We found only one-third of published psychology research is reliable – now what?

#24

Post by Lawtonfogle »

subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

I went to college and worked on both degrees in Computer Science and Psychology. I wanted to go into researching in a field that crossed the two of them. I even did undergraduate research in a lab that worked at creating and testing VR to aid in therapy.

What I saw, read, and heard as I began my climb up the Ivory Tower made me turn back and go into industry instead. Once you are inside you begin to see the corruption. From the lack of pay to people who were allowing their views to corrupt their work (often seen by tweaking definitions of terms, especially in the realms of sociology and social psychology).

I still want to go back, but I would like to do so once I'm personally financially secure as that would remove one of the vectors for corruption.

P.S. To be clear, nothing was bad in the lab I worked in, perhaps because there was little to make political about creating VR landscapes and interactions.

Re: We found only one-third of published psychology research is reliable – now what?

#25

Post by sanoli »

subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

> Now that science is proven to be systemically corrupt, what will "rational" people base their understanding on?

Oh yes, and all the scientific advancements will stop working from now on since science is proven to be corrupt. I can hear the satellites and the MRI machines crashing because science doesn't work anymore...

What this means is that studies are less rigorous than promoted to be.

Re: We found only one-third of published psychology research is reliable – now what?

#26

Post by nonbel »

Now what? You need to attempt independent replications of every published claim going forward. It is clear a single published result is unreliable. This is well known but apparently needed to be rediscovered by those who misunderstand the meaning of a p-value.

>"The first is a p-value, which estimates the probability that the result was arrived at purely by chance and is a false positive. (Technically, the p-value is the chance that the result, or a stronger result, would have occurred even when there was no real effect.) Generally, if a statistical test shows that the p-value is lower than 5%, the study’s results are considered “significant” – most likely due to actual effects."

No, there is endless literature on this. Just to start:

http://andrewgelman.com/2015/07/21/a-bad-definition-of-stati...

Re: We found only one-third of published psychology research is reliable – now what?

#27

Post by eru »

rjruxhchd wrote: post #21

Far more worrying is that only a third of economics findings can be replicated without the author's help, and yet these are the supposedly scientific findings our politicians insist on basing policy off of. Even if policy seems counterintuitive and clearly harmful to the general public, we're given the explanation "because economics says so." It's clear now from the Federal Reserve replicability study that we're bein…

And if I read that right, they were just trying to run the code provided with the papers again.

Re: We found only one-third of published psychology research is reliable – now what?

#28

Post by Lawtonfogle »

omginternets wrote: post #6

I actually have an answer for this: Stop talking about "psychology" like it's a monolithic thing. Start by separating "hard" psychology (perception, attention, psychophysics, etc) from social psychology. Papers about saccadic adaptation are, I suspect, much more reproducible than those about implicit association.

One thing I noticed is that the more focused the subset of psychology, the better the results. For example psychology that looks at a group of neurons and how they function is vastly better than psychology looking at how some early life event X leads to later problems in life.

Re: We found only one-third of published psychology research is reliable – now what?

#29

Post by Lawtonfogle »

GalacticDomin8r wrote: post #23
subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

> Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Yup, and now that the science in question was attempted to be verified and shown to be poor, we can improve the process. Try that with religion. > Now that science is proven to be systemically corrupt Huh? Anyone who took psychological research as gospel does so at their own risk and ignorance. We've l…

>Yup, and now that the science in question was attempted to be verified and shown to be poor, we can improve the process. Try that with religion.

Can we?

To the scientists devoted to furthering knowledge, they will improve. But there will be many who hold onto this. A subset of what we currently see as science is probably better described as religion already and we are soon to see this distinction clearer than in the past.

Re: We found only one-third of published psychology research is reliable – now what?

#30

Post by subliminalzen »

sanoli wrote: post #25
subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

> Now that science is proven to be systemically corrupt, what will "rational" people base their understanding on? Oh yes, and all the scientific advancements will stop working from now on since science is proven to be corrupt. I can hear the satellites and the MRI machines crashing because science doesn't work anymore... What this means is that studies are less rigorous than promoted to be.

The signal to noise ratio isn't good though. Personal integrity has been sacrificed for scientific "advancement".
Post reply on HN