Live data from Hacker News

Rebuilding after the replication crisis

asteriskmag.com

21–30 of 76 posts

Re: Rebuilding after the replication crisis

#21
Statistical research based psychology is hardly a real science. It has no feedback from reality. It has nothing whose success in the real world depends on the accuracy of the research.

Clinical psychology is still useful in my opinion. It still helps people. There's a real feedback loop where understanding can change outcomes.

I dislike the falsifiability approach (if its falsifiable it's science) and the peer review attitude (the peer review process and scientific consensus is what defines science).

My approach is that you need to close a loop. You need to do something useful whose success depended on the truthfulness of the research, and only to the extent of this dependence was anything proved.

Re: Rebuilding after the replication crisis

#22
post #11
post #8

Earlier quoted context omitted.

got dumped by a psychologist?

Dude. Psychologist still study Froyd. That's like doctors studying phrenology or physicst studying phlogiston. Outside of a history lesson of what not to do, it's not really useful.

At least in my University, in 2010, they were very adamant to let us know in our first month there that Freud was quite wrong and that this is not what modern Psychology is like.

If you think Freud is well-accepted you might be getting your views on Psychologists based on TV shows, or at minimum are vastly overgeneralizing.

Re: Rebuilding after the replication crisis

#23

How could any replication study sample from the same pattern of people, given that there are so many cultures with different ways of living, eating, spending time etc. All of these cultures also constantly change. Universal representativity is unattainable, so any replication will have a number of unknown sample variables that changed. This is still not highlighted enough in most papers I read.

Even producing results that reliably replicate across "American psychology college students age 18-21" would be a start.

Re: Rebuilding after the replication crisis

#24

How could any replication study sample from the same pattern of people, given that there are so many cultures with different ways of living, eating, spending time etc. All of these cultures also constantly change. Universal representativity is unattainable, so any replication will have a number of unknown sample variables that changed. This is still not highlighted enough in most papers I read.

That's the whole point of sampling. If you get a big enough sample, and you have a reasonably fair way of getting it, then all those things wash out as individual variation.

Re: Rebuilding after the replication crisis

#25
post #15

Earlier quoted context omitted.

What is the significance of Amy cuddy to this discussion?

Per her Wikipedia [0], she authored papers about the debunked power posing. > The theory is often cited as an example of the replication crisis in psychology, in which initially seductive theories cannot be replicated in follow-up experiments. [0]: https://en.m.wikipedia.org/wiki/Amy_Cuddy

And is still making money off of motivational speaking based on it.

Re: Rebuilding after the replication crisis

#26

Statistical research based psychology is hardly a real science. It has no feedback from reality. It has nothing whose success in the real world depends on the accuracy of the research. Clinical psychology is still useful in my opinion. It still helps people. There's a real feedback loop where understanding can change outcomes. I dislike the falsifiability approach (if its falsifiable it's science) and the peer review…

> Clinical psychology is still useful in my opinion. It still helps people. There's a real feedback loop where understanding can change outcomes.

Things psychological research has taught us about clinical psychology; the therapeutic relationship is the only thing that reliably predicts helping the client. Not the school of thought, not training, whether you vibe with the therapist.

Re: Rebuilding after the replication crisis

#27

Statistical research based psychology is hardly a real science. It has no feedback from reality. It has nothing whose success in the real world depends on the accuracy of the research. Clinical psychology is still useful in my opinion. It still helps people. There's a real feedback loop where understanding can change outcomes. I dislike the falsifiability approach (if its falsifiable it's science) and the peer review…

In fundamental research you don't typically know the potential applications where it will be useful. In fact I dislike the current overfocus on applicability, your research can't get media-boosted unless you somehow say it's a step in curing cancer or solving climate change. Similarly, pure math is valuable even without foreseeable uses (if pressed, they will say cryptography etc. but they shouldn't have to).

Curiosity to understand how the world works is enough as motivation in science. Applications can also provide ideas but then you can also diverge from them and follow directions that look intrinsically interesting or perplexing.

What you note is not addressing the core problem, namely that academia selects for certain non-diverse personality traits, like very high conscientious ess, wanting to please authority, tolerance of monotony etc which all lead to it becoming an inside baseball, working to scratch each other's backs.

You want to force them to turn away from their navel gazing by orienting them to concrete useful applications but what you really should incentivize is research whose main characteristic isn't that it's useful or all the current trendy hivemind consensus thinks it's what you're supposed to do, but the pursuit of understanding, the desire to see clearly and get closer to truth. And this doesn't just depend on what you incentivize with metrics but also what personalities are allowed in in the first place.

Re: Rebuilding after the replication crisis

#28

>One of my formative experiences as a PhD student, in 2011, was submitting a replication study to the Journal of Personality and Social Psychology, only to be told that the journal did not publish replications under any circumstances (you might be thinking, “WTF?” — and we were too). A while back, I asked a psychology professor why replication studies were frowned upon. She said something like "studies need to be uni…

not only social sciences. Except for the very visible areas of ML, the same happens in actual science and engineering...

Example: the thousands of fraudulent XRD spectra of made-up compounds.

Re: Rebuilding after the replication crisis

#29

Statistical research based psychology is hardly a real science. It has no feedback from reality. It has nothing whose success in the real world depends on the accuracy of the research. Clinical psychology is still useful in my opinion. It still helps people. There's a real feedback loop where understanding can change outcomes. I dislike the falsifiability approach (if its falsifiable it's science) and the peer review…

IMO, this is why game-designers, sales people, actors, anthropologists (to some extent) are IMO much closer to understanding human behavior than social psychologists.

Re: Rebuilding after the replication crisis

#30

>One of my formative experiences as a PhD student, in 2011, was submitting a replication study to the Journal of Personality and Social Psychology, only to be told that the journal did not publish replications under any circumstances (you might be thinking, “WTF?” — and we were too). A while back, I asked a psychology professor why replication studies were frowned upon. She said something like "studies need to be uni…

It probably takes a different type of academic to do replication studies. It's a specific kind of detective work, to see if you can expose someone else's mistakes which are sometimes even deliberate.

We need more of this type of people.

EDIT: In addition to this, perhaps we should stimulate new students to do a replication study as part of their education.

Post reply on HN