Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We found only one-third of published psychology research is reliable – now what?

theconversation.com

Post reply on HN
Print view View original

11–20 of 74 posts

Re: We found only one-third of published psychology research is reliable – now what?

#11

Post by unprepare »

vox_mollis wrote: post #3

Sneaking suspicion that this will readily extend to other soft sciences, particularly sociology. A good chunk of our social and educational policy is informed by psychology and sociology. Since we have all grown up within various systems and institutions that have been largely informed by bad research, it's unlikely much will happen, since we see the claims made by these charlatans to be normal or common sense. Nonet…

This seems to me to be a clear cut case of poor incentives in the academic research sector.

No graduate student is incentivized to re perform existing experiments to verify them, no they find much more success in creating a unique experiment which tests a previously untested hypothesis.

This is modern academic science. If you aren't doing something newsworthy, you're not getting any funding.

Re: We found only one-third of published psychology research is reliable – now what?

#12

Post by DennisP »

A friend of mine is a social psychology researcher, who's published work destroying the statistics in other papers. He says this claim of unreliability is a lot more dubious than the media portrays.

In many cases, they say a finding was not reproduced simply because the new study didn't quite meet the cutoff for "significance." However, if you do a meta-analysis of both studies, you find that the likelihood of the effect being real is higher, rather than lower.

In other cases, the replications were done poorly. The first step of many studies is calibration. You do tests to find out what sort of things your subjects are familiar with, and calibrate questions accordingly. A common mistake in the replications was to skip the calibration step, and simply reuse the questions on subjects with different backgrounds.

Re: We found only one-third of published psychology research is reliable – now what?

#13

Post by subliminalzen »

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1]

Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony.

Now that science is proven to be systemically corrupt, what will "rational" people base their understanding on?

Aside: I designed a personality test / psychoanalytical tool that was inspired by Carl Jung. It's called Critical Stimulus and it can be found at https://www.thegamecrafter.com/games/critical-stimulus. The printable version can be downloaded at gumroad: https://gumroad.com/l/criticalstimulus

.

[1] http://www.economist.com/news/briefing/21588057-scientists-t...

[2] https://en.wikipedia.org/wiki/Self_in_Jungian_psychology

Re: We found only one-third of published psychology research is reliable – now what?

#14

Post by omginternets »

hga wrote: post #10
simonster wrote: post #8

Earlier quoted context omitted.

The OSC Science paper separates cognitive and social psychology papers and journals, and cognitive psychology papers replicate at about twice the rate of social psychology papers. But still only a bit more than half of the cognitive papers replicate, so I think there are still deeper issues.

But still only a bit more than half of the cognitive papers replicate, so I think there are still deeper issues. That puts them in the general level of quality for all bio-medical research, so, yeah, deeper issues indeed, but I find it fascinating that this sub-group is so good.

This because that sub-group is very, very close to medical research.

The problem with social psychology, on the whole, is that it's highly politicized. This is especially true with regards to any study involving race, gender, or "identity" (in the sense of identity politics).

Re: We found only one-third of published psychology research is reliable – now what?

#15

Post by omginternets »

subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

>Now that science is proven to be systemically corrupt, what will "rational" people base their understanding on?

This is not a new notion in the least. The "rational" people will continue doing what they've always done: revising their conclusions.

Science is a process, and while we can question the notion of "scientific progress" in philosophical terms, this is neither a new idea nor evidence that science doesn't work. It's certainly not evidence that science is no better than irrational thinking.

>Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.

Few people take psychoanalysts seriously these days, in large part because of their long record of absurd claims and shoddy clinical work. Jungian theory has it's place in a conversation about literary theory, but not in a conversation about science.

Re: We found only one-third of published psychology research is reliable – now what?

#16

Post by omginternets »

vox_mollis wrote: post #3

Sneaking suspicion that this will readily extend to other soft sciences, particularly sociology. A good chunk of our social and educational policy is informed by psychology and sociology. Since we have all grown up within various systems and institutions that have been largely informed by bad research, it's unlikely much will happen, since we see the claims made by these charlatans to be normal or common sense. Nonet…

>particularly sociology

I'll take it a step further. The reason we're hearing about psychology and not sociology is that few people take sociology seriously, as a science.

Sociology is a science like political science is a science in that any irrefutable claim constitutes a valid theory. Evidence is secondary and the notion of falsifiability doesn't even come up.

Psychology has a very hard-science core, whose reputation is so tarnished by its "social" counterpart that no self-respecting researcher wants to be called a psychologist anymore. Replace your searches for psychology with cognitive science and you'll find a lot of good, reproducible, research such as this: http://cavlab.net/?lang=en

Re: We found only one-third of published psychology research is reliable – now what?

#17

Post by weland »

subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

> Only a quarter of scientific drug research is successfully reproduced as well.[1]

This is a debate in semantics: research that is not successfully reproduced is not scientific.

It should be part of that which society frowns down on as non-verifiable testimony.

Natural sciences or engineering-related research that cannot be reproduced is nothing but scientific theater. There are various reasons why people do that (including financial ones in the drug industry), but that's a discussion for another time.

This is not proof that "scientific rationality" is brainwashing people. If anything, it's proof that non-reproducible research dressed as legitimate science is dangerous -- so dangerous, in fact, that it can put lives in danger (e.g. when it happens in drug research).

Re: We found only one-third of published psychology research is reliable – now what?

#18

Post by [deleted] »

subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

[deleted]

Re: We found only one-third of published psychology research is reliable – now what?

#19

Post by omginternets »

weland wrote: post #17
subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

> Only a quarter of scientific drug research is successfully reproduced as well.[1] This is a debate in semantics: research that is not successfully reproduced is not scientific . It should be part of that which society frowns down on as non-verifiable testimony. Natural sciences or engineering-related research that cannot be reproduced is nothing but scientific theater. There are various reasons why people do that (…

>This is a debate in semantics: research that is not successfully reproduced is not scientific.

That's both untrue and fallacious (see: no true Scotsman).

If you set your p-value threshold at .05, then one in twenty experiments will produce a false positive. As such, plenty of research is conducted in a benevolent and meticulous fashion, only to yield a non-reproducible result. It's still scientific; it's just not true.

I don't agree with subliminalzen, but (with all due respect -- really!) your comment is hogwash.

Re: We found only one-third of published psychology research is reliable – now what?

#20

Post by [deleted] »

omginternets wrote: post #15
subliminalzen wrote: post #13

It's not just psychological research that is deeply flawed. Only a quarter of scientific drug research is successfully reproduced as well.[1] Carl Jung claimed one of the chief factors responsible for mass brainwashing is scientific rationality.[2] Society worships the Goddess of Reason while frowning down on "irrational" and non-verifiable religious testimony. Now that science is proven to be systemically corrupt, w…

>Now that science is proven to be systemically corrupt, what will "rational" people base their understanding on? This is not a new notion in the least. The "rational" people will continue doing what they've always done: revising their conclusions. Science is a process, and while we can question the notion of "scientific progress" in philosophical terms, this is neither a new idea nor evidence that science doesn't wor…

[deleted]
Post reply on HN