Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If you think psychological science is bad, imagine how bad it was in 1999

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

21–30 of 238 posts

Re: If you think psychological science is bad, imagine how bad it was in 1999

#21

Post by logicchains »

m12k wrote: post #15

When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science?

>When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science?

In a sense we do have this: engineering and finance. Engineering turns good hard science into new tools, machines and weapons, and Finance turns good (predictive) soft science into new ways to make money.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#22

Post by antiterra »

The part that confused me the most, as someone who has never done research science, was how the questions were casually categorized as gender, ethnicity or no-identity salient.

If someone is from Brazil and is of second or third generation Japanese descent, how much of the questions are 'salient' to Brazilian identity vs Japanese? There's an unspoken implication that part of the 'good at math' stereotype relies to some degree on speaking a non-English language at home, which I don't think is a safe assumption at all.

> In the Asian identity-salient condition, participants (n = 16) were asked (a) whether their parents or grandparents spoke any languages other than English, (b) what languages they knew, (c) what languages they spoke at home, (d) what opportunities they had to speak other languages on campus, (c) what percentage of these opportunities were found in their residence halls, and (f) how many generations of their family had lived in America.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#23

Post by NortySpock »

mellavora wrote: post #17
burnte wrote: post #13

Earlier quoted context omitted.

Religious biases don't count as valid moral objections. Religion has a strong negative effect on critical thinking skills (1, 2). If you dogs/pigs/shellfish/mixed-fabrics are inherently unclean or that women are property, your objections to a dating website are probably not going to have seriousl scientific ramifications. 1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5742220/ 2. https://www.google.com/search?q=Reli…

Ok, I have some reason to agree with you, but if we rule out religion as a valid basis for morality, what do you suggest we replace it with?

Act or Rule Utilitarianism.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#24

Post by munificent »

m12k wrote: post #15

When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science?

> it's insane that metascience isn't studied more.

It is, probably more than is apparent. See:

https://en.wikipedia.org/wiki/Philosophy_of_science

https://en.wikipedia.org/wiki/Sociology_of_scientific_knowle...

There is an interesting recursive problem here, though: what tools do you use to scientifically analyze the scientific process? Whatever tools you use will themselves be hobbled by the same systemic flaws you are trying to understand.

Also, as any sociologist will be happy to tell you, incentive structures and other human group behavior gets in the way. It's probably hard to get funding for a study that shows that all the other departments at your university aren't quite the flawless seekers of truth they appear to be.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#25

Post by throwawaysea »

m12k wrote: post #15

When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science?

There was a past discussion about peer review on HN: https://news.ycombinator.com/item?id=20607259

Another related discussion was about the grievance studies scandal, which also touches on peer review and academic rigor in journals: https://news.ycombinator.com/item?id=18127811

Re: If you think psychological science is bad, imagine how bad it was in 1999

#26

Post by publicola1990 »

Much of such dubious "science" is packaged by self help authours as miracle principles which are going to set ones life right. Popular books like "Power Of Habit" and Cal Newport's books also seem to fall into this category, and seem to enjoy huge patronage, even in HN.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#27

Post by kizer »

Interesting how as I began reading the excerpts from the paper I had to stop myself and realize that the author himself had ironically conferred a bias against the paper, LOL. I was primed to find all the flaws.

I think psychology and sociology are legitimate and worthy studies, but they run into issues with the scientific method itself due to the ambiguous and “high-level” nature of their concepts and theories. It’s hard to create meaningful, repeatable experiments. So perhaps it should be emphasized how important it is to put effort into constructing experiment... and in particular keeping the subjects unaware of what it being tested. There are probably many great examples of experiments done.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#28

Post by hannob »

ryanmarsh wrote: post #20
m12k wrote: post #15

When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science?

Similarly, I heard someone the other day assert that no one had done an double blind placebo controlled trial of the effects of FDA regulation.

That is probably true, but do you have any proposal how such a trial should be done?

RCTs are good when they can be done and I'm all for doing more of them and too often there's no good excuse for not doing them. But at some level things just get impractical.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#29

Post by hpoe »

logicchains wrote: post #21
m12k wrote: post #15

When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science?

>When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science? In a sense we do have this: engineering and finance. Engineering turns good hard science into new tools, machines and weapons, and Finance turns…

I think this is an important point, ultimately good science will produce, verifiable, testable, actionable results. Until you have that no matter how much math you use, how many lab coats you've got, no matter how many journals you publish in you're just sitting there playing with strings.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#30

Post by bluetomcat »

m12k wrote: post #15

When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science?

"People" sciences like sociology, psychology and economics can make incredibly misleading claims because one experiment over a small sample of people at a certain moment in time might seem to support a claim, while the actual reason for the observed results is a factor which is never taken in consideration. On the other hand, conducting those experiments over wider demographics and in different points in time means that the study wants to build "universal" models of how each single person in the whole world acts, which is utterly dismissive of the specific local environment around people.

Sociology in particular should always be approached highly critically, because applying those theories and reasoning in its terms often means mass control over people's free will.

Post reply on HN