Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If you think psychological science is bad, imagine how bad it was in 1999

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

231–238 of 238 posts

Re: If you think psychological science is bad, imagine how bad it was in 1999

#231

Post by dash2 »

peoplefromibiza wrote: post #128
dash2 wrote: post #47

Earlier quoted context omitted.

I think you are naive about history. I'm an economist. If I threw away the half of the data that didn't support my findings, and got caught, I'd lose my job and never publish again. I'm pretty sure the same is true in other social sciences, such as psychology. This is true irrespective of the well-documented problems that the article describes, which certainly also apply in economics and elsewhere, to varying degrees…

But let's not pretend that historians influence the policy makers as much as sociologists and economists do either. The damages when they are wrong are orders of magnitude bigger. They are assumed to be right, sometimes even without proof, until they are tragically proven wrong. And nobody lose their job anyway. Have you ever seen a sociologist lose the job because proposed something to a politician that resulted in…

I think you underestimate how influential historians are in the long run, by changing how we see ourselves. But in any case, my point was about which disciplines we can trust, not about which are more or less powerful.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#232

Post by dash2 »

whakim wrote: post #190
dash2 wrote: post #47

Earlier quoted context omitted.

I think you are naive about history. I'm an economist. If I threw away the half of the data that didn't support my findings, and got caught, I'd lose my job and never publish again. I'm pretty sure the same is true in other social sciences, such as psychology. This is true irrespective of the well-documented problems that the article describes, which certainly also apply in economics and elsewhere, to varying degrees…

The thing about textual evidence is that you can't cite an entire text (obviously). You have to selectively choose what to quote in order to support your claims. Additionally, people can write one thing, and then write other contradictory things. Or they can act in ways that contradict what they write. It is from this totality of evidence that non-quantitative methods draw their conclusions. To get to the point, I'm…

The JEL review which I quote in the linked blog certainly had read the book, and called it "replete with significantly flawed arguments, misplaced citations, and dubious conjectures". And if cutting sentences in half, to remove something which directly contradicts your thesis, doesn't count as historical malpractice, then what would?

Re: If you think psychological science is bad, imagine how bad it was in 1999

#233

Post by ouid »

SV_BubbleTime wrote: post #170
ouid wrote: post #51

Earlier quoted context omitted.

Bonferroni effects have, at least in the popular consciousnes, become much more front and center. I wonder how many people on the stre can define p-hacking. Most educated people know about the reproducibility crisis, and I would wager that around half of them understand the cause. That seems like it might create a better environment.

> around half of them understand the cause I’m in the half that doesn’t know, apparently. I considered that with hard sciences, the cost, time, equipment, conditions, etc were limiting factors. I think this paper on subatomic particles changing into antimatter is interesting, it was observed in a unique facility, once, under some condition that can never repeat, etc. Just kinda have to take your word for it. As to so…

Bonferroni effects. In the parent article referred to as researcher degrees of freedom, in pop culture referred to as p-hacking. In machine learning, it's referred to as overfitting. They are all the same phenomenon, of having more than one hypothesis, and then using statistical measures that assume that you only have one hypothesis to claim confidence. Journals only publishing positive results is an example of this phenomenon. Changing your hypothesis to match your data is another such example.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#234

Post by burnte »

mellavora wrote: post #17
burnte wrote: post #13

Earlier quoted context omitted.

Religious biases don't count as valid moral objections. Religion has a strong negative effect on critical thinking skills (1, 2). If you dogs/pigs/shellfish/mixed-fabrics are inherently unclean or that women are property, your objections to a dating website are probably not going to have seriousl scientific ramifications. 1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5742220/ 2. https://www.google.com/search?q=Reli…

Ok, I have some reason to agree with you, but if we rule out religion as a valid basis for morality, what do you suggest we replace it with?

I'm not even sayign religion can't be a basis for morality, I'm just saying religious BIASES aren't valid arguments. Don't kill/murder/steal, those are great, clear lines about interacting with others. There's a clear benefit to all for those rules. Don't turn on your oven on saturdays, pet dogs, or cut your beard, those aren't widely applicable.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#235

Post by [deleted] »

mellavora wrote: post #17
burnte wrote: post #13

Earlier quoted context omitted.

Religious biases don't count as valid moral objections. Religion has a strong negative effect on critical thinking skills (1, 2). If you dogs/pigs/shellfish/mixed-fabrics are inherently unclean or that women are property, your objections to a dating website are probably not going to have seriousl scientific ramifications. 1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5742220/ 2. https://www.google.com/search?q=Reli…

Ok, I have some reason to agree with you, but if we rule out religion as a valid basis for morality, what do you suggest we replace it with?

[deleted]

Re: If you think psychological science is bad, imagine how bad it was in 1999

#236

Post by LudwigNagasena »

twhb wrote: post #211
LudwigNagasena wrote: post #200

Earlier quoted context omitted.

1. This is pretty trivial. Why do you think most scientists aren’t aware of their limitations? 2. This is studied in statistics classes. It is impossible to analyze data without a statistical model, so of course your conclusions rely on it. So this is also quite trivial. 3. Again, are scientists oblivious to this simple notion? I doubt it.

Why is it that mirrors flip your image left to right, but not top to bottom? That’s a question that Richard Feynman supposedly asked his grad students. Once you answer it, I think you’ll realize that #1 is anything but trivial. Answer below, stop reading this comment if you want to figure it out on your own. It’s because you’re comparing the mirror image to what you’d look like if you walked around the mirror, instea…

I fail to see how it in any way relates to #1.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#237

Post by Delk »

derbOac wrote: post #228
Delk wrote: post #191

Earlier quoted context omitted.

What makes that even more absurd in my books (apart from fostering an unhealthy academic culture) is that the purpose of a doctoral dissertation is essentially to show that you can do proper original research. Getting a null result doesn't invalidate that in any way.

Exactly. In our area, the dissertation itself and defense were evaluated on the presentation of the research, not on the results, provided that they followed the proposed plan. If you have a committee of experts who carefully evaluate a proposal and decide it's good, the results are as they are. Broadening the discussion a bit, it seems one feature of science, as opposed to, say philosophy, is that the conclusions re…

> it seems one feature of science, as opposed to, say philosophy, is that the conclusions regarding a hypothesis are not knowable a priori.

That should be a defining characteristic of any academic inquiry, regardless of whether it's science or not.

I have no training in non-quantitative fields, and my academic experience is from CS where the "science" part is often so-so. As such this should be taken partially as a layman view. However, my impression is that while in non-scientific academic fields the research isn't necessarily taking the form of explicit hypothesis testing, more or less similar criteria for intellectual inquiry should apply.

The research might be more about observation and critical (often non-quantitative and non-absolute) evaluation of arguments, and as such the validity of the methods (such as whether the hypothesis is assumed or genuinely questioned) might not always be as easy to judge [1].

The process might not be as easily formalized or judged as in science, but the mentality of critical inquiry should be similar. If the hypothesis is assumed and not questioned, that's no longer any kind of academic inquiry. It becomes politics, in the pejorative sense.

> I think in contemporary academics there's some implicit idea that the quality of a researcher lies in their ability to identify hypotheses that are "correct", as opposed to simply following through with good but ultimately "incorrect" hypotheses.

I think that's partially just psychology and human nature. We like results that make us directly know (or think we know) more, and results that basically tell us we still don't know less. Few people like uncertainty.

The society outside of the academia certainly values the former more than the latter, and funding and other external incentives probably exacerbate the underappreciation of negative results.

[1] Or perhaps it is, to an expert, but having that judgment would require the kind of experience in those fields that I don't have.

Re: If you think psychological science is bad, imagine how bad it was in 1999

#238

Post by whakim »

dash2 wrote: post #232
whakim wrote: post #190

Earlier quoted context omitted.

The thing about textual evidence is that you can't cite an entire text (obviously). You have to selectively choose what to quote in order to support your claims. Additionally, people can write one thing, and then write other contradictory things. Or they can act in ways that contradict what they write. It is from this totality of evidence that non-quantitative methods draw their conclusions. To get to the point, I'm…

The JEL review which I quote in the linked blog certainly had read the book, and called it "replete with significantly flawed arguments, misplaced citations, and dubious conjectures". And if cutting sentences in half, to remove something which directly contradicts your thesis, doesn't count as historical malpractice, then what would?

It's just not that simple. I'm not making a value judgment on the book (I haven't read it), but a person can say two things in the same sentence and the broader context can make it clear that they're just covering their ass, for example. Perhaps that's not what's going on here. Perhaps the book does constitute "malpractice." But...I think the situation is more complex than you're giving it credit for, and I wouldn't be comfortable drawing conclusions without a greater familiarity with the book and the responses to it. I also don't give a lot of credence to the blog you linked, since they use (as one of their two pieces of evidence) a critique which openly admits it hasn't actually read the thing that is being critiqued.

To your last point, plagiarism, for example, definitely counts as malpractice and humanities professors lose their jobs for it.

Post reply on HN