Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If you think psychological science is bad, imagine how bad it was in 1999

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

111–120 of 238 posts

Re: If you think psychological science is bad, imagine how bad it was in 1999

#111

Post by aeturnum »

m12k wrote: post #15

When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science?

If you are interested in this I really recommend you look at writings from the fields that have descended from what was called "Laboratory Studies" and now variously calls itself "Science studies", or "science and technology studies" or "science, technology and society" (STS is a common acronym). For the stuff that's very lab-focused I'd say you could start with Bruno Latour and Steven Shapin, both classics of the older guard of the field.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#112

Post by aeturnum »

libraryofbabel wrote: post #38
m12k wrote: post #15

When you think about it, it's insane that metascience isn't studied more. We're going to question every little detail about the universe, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" science?

There is a ton of current and past scholarship on the sociology, history, and philosophy of science, and peer-review and publication is actually a pretty hot topic in those fields. Although yes, perhaps it would be nice if that work was better funded, or if practicing scientists paid more attention to it rather than just repeating old myths about how science works. For example, here’s a scholarly article on the exact…

Another classic on this subject is Shapin's "Pump and Circumstance" - which is also freely available:

https://scholar.harvard.edu/files/shapin/files/shapin-pump_c...

Re: If you think psychological science is bad, imagine how bad it was in 1999

#113

Post by walrus »

PatentlyDC123 wrote: post #105

Is there a method of flagging papers in scientific journals that have been criticized or refuted, e.g., by later studies or proven inability to replicate the data? In legal research services, like Lexis Nexis or Westlaw, many cases are "flagged" when a later case or statute reverses, narrows, or otherwise affects the earlier case. This system warns lawyers that they may not be able to cite the flagged case in their c…

A journal can publish a retraction.

The website Retraction Watch[1] aggregates these retractions and provides a database that you can query. Reference management software like Zotero[2] can use this to monitor your collection of papers and notify you when one is retracted.

[1] https://retractionwatch.com/

[2] https://www.zotero.org/blog/retracted-item-notifications/

Re: If you think psychological science is bad, imagine how bad it was in 1999

#114

Post by jbullock35 »

mateo411 wrote: post #94
enkid wrote: post #8

Earlier quoted context omitted.

Tangent, but the Stanford Prison Experiment is an awful example of science. It was a researcher who wanted to prove a point and created the conditions to collect the data to prove that point. I hate that it's often the only psychological experiment many people are familiar with.

What about the Milgram experiment? This is also a very well known psychological experiment. Was the science behind the Milgram experiment rigorous?

Milgram's obedience studies didn't involve randomized treatments, and he had small numbers of subjects (typically around 40) in each of his many conditions. On the other hand, Milgram's investigations were serious, systematic, and in good faith -- which makes them worlds better than the Stanford Prison Experiment.

Another reply in this thread suggests that "a large number of participants may have been aware that the actor wasn't really suffering when they administered the punishment." I've studied the topic and found no evidence of this point. In addition, the claim is hard to square with many subjects' reactions -- for example, their nervous laughter and their frequent protests, even as they continued to deliver what they thought were harmful electric shocks.

See https://news.ycombinator.com/item?id=25928569 for more on efforts to replicate Milgram's results.

Tl;dr: there is no equivalence between the Stanford Prison Experiment and Milgram's work on obedience. Milgram's work was superior.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#115

Post by rajin444 »

bfors wrote: post #97
failwhaleshark wrote: post #73

Earlier quoted context omitted.

You can construct proofs for mathematics, but not for the existence or refutation of a divine creator.

True, but with math you also eventually get down to axioms that can't be proven.

I don't think we know if we can know everything or not. That seems like _the_ question to answer.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#116

Post by jedberg »

I studied CogSci in the late 90s, which involved taking psychology courses. Every psych course that ended in zero required you to participate in psychology experiments.

The profs like to say that 18-22, mostly white/asian, mostly rich kids are the most studied group of people in the USA.

Or course we now know that a ton of psychology research doesn't actually apply to people outside that small narrow window of people.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#117

Post by hypersoar »

PatentlyDC123 wrote: post #105

Is there a method of flagging papers in scientific journals that have been criticized or refuted, e.g., by later studies or proven inability to replicate the data? In legal research services, like Lexis Nexis or Westlaw, many cases are "flagged" when a later case or statute reverses, narrows, or otherwise affects the earlier case. This system warns lawyers that they may not be able to cite the flagged case in their c…

An issue that comes to mind is, ironically, authority. Who decides when a paper has been discredited? I can see all sorts of incentive problems with such a system. On the other hand, Westlaw is only cataloging what has already been decided. If the Supreme Court overturns a prior case, then it's overturned whether you agree with the reasoning or not.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#118

Post by thaumasiotes »

jedberg wrote: post #116

I studied CogSci in the late 90s, which involved taking psychology courses. Every psych course that ended in zero required you to participate in psychology experiments. The profs like to say that 18-22, mostly white/asian, mostly rich kids are the most studied group of people in the USA. Or course we now know that a ton of psychology research doesn't actually apply to people outside that small narrow window of people…

It doesn't apply within that window either.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#119

Post by tern »

publicola1990 wrote: post #26

Much of such dubious "science" is packaged by self help authours as miracle principles which are going to set ones life right. Popular books like "Power Of Habit" and Cal Newport's books also seem to fall into this category, and seem to enjoy huge patronage, even in HN.

Who do you look to for wisdom about how to live your life?

Re: If you think psychological science is bad, imagine how bad it was in 1999

#120

Post by hypersoar »

stevetodd wrote: post #46

Reading the other comments about the lack of integrity in scientific research that they’ve witnessed, is it any wonder why people might be skeptical about important issues like climate change, vaccines, masks, etc? There’s great research being done and a larger part of the population is becoming more and more skeptical because lies are being published and touted. Science needs to clean its house.

The skepticism about climate change is due much less to issues with the scientific community that than it is to decades of propoganda funded by people who fear losing money to efforts to fight it.
Post reply on HN