Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We found only one-third of published psychology research is reliable – now what?

theconversation.com

Post reply on HN
Print view View original

31–40 of 74 posts

Re: We found only one-third of published psychology research is reliable – now what?

#31

Post by drumdance »

Lawtonfogle wrote: post #28
omginternets wrote: post #6

I actually have an answer for this: Stop talking about "psychology" like it's a monolithic thing. Start by separating "hard" psychology (perception, attention, psychophysics, etc) from social psychology. Papers about saccadic adaptation are, I suspect, much more reproducible than those about implicit association.

One thing I noticed is that the more focused the subset of psychology, the better the results. For example psychology that looks at a group of neurons and how they function is vastly better than psychology looking at how some early life event X leads to later problems in life.

Isn't that really neuroscience?

Re: We found only one-third of published psychology research is reliable – now what?

#32

Post by GalacticDomin8r »

Lawtonfogle wrote: post #29
GalacticDomin8r wrote: post #23

Earlier quoted context omitted.

> Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Yup, and now that the science in question was attempted to be verified and shown to be poor, we can improve the process. Try that with religion. > Now that science is proven to be systemically corrupt Huh? Anyone who took psychological research as gospel does so at their own risk and ignorance. We've l…

>Yup, and now that the science in question was attempted to be verified and shown to be poor, we can improve the process. Try that with religion. Can we? To the scientists devoted to furthering knowledge, they will improve. But there will be many who hold onto this. A subset of what we currently see as science is probably better described as religion already and we are soon to see this distinction clearer than in the…

> Can we?

If we couldn't, this HN discussion wouldn't exist.

Re: We found only one-third of published psychology research is reliable – now what?

#33

Post by nonbel »

geomark wrote: post #22

Let's be clear on the differencen between reproducible and replication . Reproducible is when an experiment's data and methods are published and other researchers can reproduce the same statistical results. An experiment that has been replicated is a new experiment that applies the same methods to a new set of subjects and arrives at statistically equivalent results. The distinction is important. Reproducible researc…

>"Let's be clear on the difference between reproducible and replication."

I did not find this clear. What is the distinction you are trying to make?

Re: We found only one-third of published psychology research is reliable – now what?

#34

Post by drumdance »

omginternets wrote: post #15
subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

>Now that science is proven to be systemically corrupt, what will "rational" people base their understanding on? This is not a new notion in the least. The "rational" people will continue doing what they've always done: revising their conclusions. Science is a process, and while we can question the notion of "scientific progress" in philosophical terms, this is neither a new idea nor evidence that science doesn't wor…

I don't know about psychoanalysis as defined by Freud and Jung, but therapy in general is an intensely personal experience.

I'm sure there are scientific aspects that can be brought to bear on a situation, but for a lot of people the "literary theory" part is just as helpful. Especially when you stop to think how much neurosis is fueled by pop culture (i.e. status envy).

Re: We found only one-third of published psychology research is reliable – now what?

#35

Post by drumdance »

Lawtonfogle wrote: post #24
subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

I went to college and worked on both degrees in Computer Science and Psychology. I wanted to go into researching in a field that crossed the two of them. I even did undergraduate research in a lab that worked at creating and testing VR to aid in therapy. What I saw, read, and heard as I began my climb up the Ivory Tower made me turn back and go into industry instead. Once you are inside you begin to see the corruptio…

VR and psychology -- cool! I got to visit Stanford's VR lab last year. They are doing some really cool pro-social stuff that has lots of therapeutic potential. Excited to see what comes out of it over the next 20 years.

Re: We found only one-third of published psychology research is reliable – now what?

#36

Post by weland »

omginternets wrote: post #19
weland wrote: post #17

Earlier quoted context omitted.

> Only a quarter of scientific drug research is successfully reproduced as well.[1] This is a debate in semantics: research that is not successfully reproduced is not scientific . It should be part of that which society frowns down on as non-verifiable testimony. Natural sciences or engineering-related research that cannot be reproduced is nothing but scientific theater. There are various reasons why people do that (…

>This is a debate in semantics: research that is not successfully reproduced is not scientific. That's both untrue and fallacious (see: no true Scotsman ). If you set your p -value threshold at .05, then one in twenty experiments will produce a false positive. As such, plenty of research is conducted in a benevolent and meticulous fashion, only to yield a non-reproducible result. It's still scientific; it's just not…

> That's both untrue and fallacious (see: no true Scotsman).

No it's not. A sound scientific approach requires that a theory be based on reproducible results. If an experiment that verifies your theory confirms your result today and infirms it tomorrow, then the theory, the experimental approach, or both, are wrong.

Of course, experiments that can't be reproduced are part of the scientific endeavour. Every discovery comes at the end of a long sequence of experiments with results scattered all over the grah. But treating them as anything other than stumbling steps that help you refine your understanding of the problem or as dead ends is as unscientific as it gets.

Re: We found only one-third of published psychology research is reliable – now what?

#37

Post by drumdance »

subliminalzen wrote: post #30
sanoli wrote: post #25

Earlier quoted context omitted.

> Now that science is proven to be systemically corrupt, what will "rational" people base their understanding on? Oh yes, and all the scientific advancements will stop working from now on since science is proven to be corrupt. I can hear the satellites and the MRI machines crashing because science doesn't work anymore... What this means is that studies are less rigorous than promoted to be.

The signal to noise ratio isn't good though. Personal integrity has been sacrificed for scientific "advancement".

It's always been that way. A good book about this is Bill Bryson's A Short History of Nearly Everything.

Re: We found only one-third of published psychology research is reliable – now what?

#38

Post by geomark »

nonbel wrote: post #33
geomark wrote: post #22

Let's be clear on the differencen between reproducible and replication . Reproducible is when an experiment's data and methods are published and other researchers can reproduce the same statistical results. An experiment that has been replicated is a new experiment that applies the same methods to a new set of subjects and arrives at statistically equivalent results. The distinction is important. Reproducible researc…

>"Let's be clear on the difference between reproducible and replication." I did not find this clear. What is the distinction you are trying to make?

Reproduce: take the published data and methods, rerun the analysis, see if you get the same results. No new experiment.

Replicate: perform a new experiment with new subjects, apply the same methods, see if you get statistically equivalent results.

Re: We found only one-third of published psychology research is reliable – now what?

#40

Post by omginternets »

drumdance wrote: post #34
omginternets wrote: post #15

Earlier quoted context omitted.

>Now that science is proven to be systemically corrupt, what will "rational" people base their understanding on? This is not a new notion in the least. The "rational" people will continue doing what they've always done: revising their conclusions. Science is a process, and while we can question the notion of "scientific progress" in philosophical terms, this is neither a new idea nor evidence that science doesn't wor…

I don't know about psychoanalysis as defined by Freud and Jung, but therapy in general is an intensely personal experience. I'm sure there are scientific aspects that can be brought to bear on a situation, but for a lot of people the "literary theory" part is just as helpful. Especially when you stop to think how much neurosis is fueled by pop culture (i.e. status envy).

>but therapy in general is an intensely personal experience.

And empirically speaking, psychoanalytic therapy has a piss-poor record in dealing with mental illness.

If you're looking for spiritual guidance, then maybe a psychoanalyst can help. If you're looking for clinical efficacy, they demonstrably don't.

I'll leave it as an exercise to the reader to make a relevant Google Scholar query, but be careful not to confuse psychoanalysis with clinical psychology.

> for a lot of people the "literary theory" part is just as helpful.

Fine, but this is a conversation about psychology as a science.

Post reply on HN