Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If you think psychological science is bad, imagine how bad it was in 1999

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

191–200 of 238 posts

Re: If you think psychological science is bad, imagine how bad it was in 1999

#191

Post by Delk »

derbOac wrote: post #124
tgv wrote: post #14

And that's without knowing how the study was really executed. Where I worked, there was a relatively successful postdoc who presented the results of his p<.05 significant pilot study. When asked, he said it wasn't the first pilot. It was the 20th.

In the department I worked for as a faculty member, there was an issue where the other areas would have graduation rates around 40-50%, as opposed to our area which was above 95% or so. This came up in the context of an external review. When people asked around, informally what was said was that the grad students in the other areas (especially one area, in the experimental molecular biosciences) would leave after hav…

What makes that even more absurd in my books (apart from fostering an unhealthy academic culture) is that the purpose of a doctoral dissertation is essentially to show that you can do proper original research.

Getting a null result doesn't invalidate that in any way.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#192

Post by [deleted] »

munificent wrote: post #164
xxpor wrote: post #141

Earlier quoted context omitted.

The only nit I'd pick with the list is: * That truth can be expressed numerically. Isn't the basic point of quantum physics that this isn't true? We can only make guesses with probabilities, but we can't know the actual truth, and therefore can't express it numerically.

Sure, but the probabilities are numbers too. Which is again sort of acknowledging the need to fit quantum mechanics into a numeric framework. Imagine you were studying ice cream flavors. You might design a study like, "We'll ask a lot of people and the flavor that the most people prefer is the best." In other words, the meta process you use to design your experiment itself tacitly assumes you need a numeric result. T…

[deleted]

Re: If you think psychological science is bad, imagine how bad it was in 1999

#193

Post by mattkrause »

mykowebhn wrote: post #16

I worked at a well-known clinical psychology lab at an Ivy League school many years ago. There was so much "gaming" of the results. For example, our PI didn't want to include a subject in our study because his scores weren't elevated enough, and our PI was worried that his score wouldn't drop enough which would adversely impact our results. Another example: our active treatment therapists knew exactly what they were…

The first example doesn't seem that bad. No intervention cannot reduce something if it isn't there to begin with.

The screening should be disclosed in the methods (and, ideally, pre-specified), but you do need to account for floor/ceiling effects somehow.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#194

Post by wolverine876 »

> If you think psychological science is bad

I know it's often repeated, but is there evidence that "psychological science" is worse (by some measure and to a significant degree) than other social sciences? As long as we're talking about science, let's look at evidence! ;)

Anyone studying human behavior, on any scale, has the added challenge that it's so complex that you can't really isolate simple mechanics, as you can in physics or chemistry. People are far more complex than a molecule, or even a billion molecules.

BTW, I'm aware of the well-known cite, the NY Times article on the 'reproducability crisis' from a few years ago. Few seem to have read the article: The results were reproduced, but many were at lower strength than the original research. That's important, but it's not like the researchers just complete missed and results were arbitrary.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#195

Post by 99_00 »

enkid wrote: post #107
99_00 wrote: post #100

Earlier quoted context omitted.

>the Stanford Prison Experiment is an awful example of science. This may be true, but I don't think the evidence you give supports your assertion. >It was a researcher who wanted to prove a point and created the conditions to collect the data to prove that point "prove a point" is the hypothesis "created the conditions" is the experiment "collect the data to prove that point" is the observation

No, an experiment should not be set up to prove a hypothesis, it should be set up to test a hypothesis. In one the hypothesis is falsifiable, and in the other it's not.

If people didn't act the way the experimenter expected wouldn't that disprove their hypothesis in this case?

Re: If you think psychological science is bad, imagine how bad it was in 1999

#196

Post by lupire »

panagathon wrote: post #189
svieira wrote: post #188

Earlier quoted context omitted.

> That truth can be expressed numerically I don't think this is what is held by those-who-do-science-and-philosophy-at-large (though it may be a generally accepted hand-wave, I don't know). See, for example Category Theory for a branch of what-I-believe-would-generally-be-called-science that doesn't use numbers, but instead expresses things with sets and relations. The logician is the intersection of the set of all s…

I think I disagree, since mathematics cannot be reduced to logic (or any other instrumentalisation). https://plato.stanford.edu/entries/philosophy-mathematics/#M...

That article says that mathematics is ill-founded, not that math has something inexpressible by logic. Mathematics can be reduced to logic that is ill-founded.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#197

Post by lupire »

cwkoss wrote: post #129
bluetomcat wrote: post #30

Earlier quoted context omitted.

"People" sciences like sociology, psychology and economics can make incredibly misleading claims because one experiment over a small sample of people at a certain moment in time might seem to support a claim, while the actual reason for the observed results is a factor which is never taken in consideration. On the other hand, conducting those experiments over wider demographics and in different points in time means t…

I majored in psychology in undergrad. A big part of why I didn't look for a psychology focused job is that the science is all so loose. I'd often learn about two different study-backed phenomena is two different classes that somewhat contradicted each other. Or I'd learn in a subsequent class that a previously taught study has been invalidated in one way or another. Almost everything is measured subjectively, so huge…

The important problems are hard. Avoiding psychology because it's messy is like the metaphor of only searching for your lost keys under a lamppost.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#198

Post by enkid »

99_00 wrote: post #195
enkid wrote: post #107

Earlier quoted context omitted.

No, an experiment should not be set up to prove a hypothesis, it should be set up to test a hypothesis. In one the hypothesis is falsifiable, and in the other it's not.

If people didn't act the way the experimenter expected wouldn't that disprove their hypothesis in this case?

No, because people did behave differently than he expected, and he coached them to behave in line with his expectations. He wasn't a neutral observer, he was a guiding force and the superintendent of the pretend prison. This is what I mean when I say it wasn't an experiment.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#199

Post by kirubakaran »

rajin444 wrote: post #115
bfors wrote: post #97

Earlier quoted context omitted.

True, but with math you also eventually get down to axioms that can't be proven.

I don't think we know if we can know everything or not. That seems like _the_ question to answer.

https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...

Re: If you think psychological science is bad, imagine how bad it was in 1999

#200

Post by LudwigNagasena »

nicoburns wrote: post #187
andi999 wrote: post #163

Earlier quoted context omitted.

Can you point to something good and useful in the Philosophie of Science? I mean let's say physics is as good as it gets (laws in mathematical language, strong experimental evidence, solid results); from my perspective a success. But it has nothing to do with how like popper imagines science. So insisting on like a popper process would also not be benificial for physics in my opinion. But probably there is better stu…

Some key things for me would be the ideas that: 1. There is no such thing as objectivity in inquiry. There is always a scientist interpreting things, and they are always looking at things through the lens of their own biases and cultural norms. The best you can hope to do is to be aware of your limitations. 1. Statistical evidence on its own is not always a good basis for believing something to be true. Typically you…

1. This is pretty trivial. Why do you think most scientists aren’t aware of their limitations?

2. This is studied in statistics classes. It is impossible to analyze data without a statistical model, so of course your conclusions rely on it. So this is also quite trivial.

3. Again, are scientists oblivious to this simple notion? I doubt it.

Post reply on HN