Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

thereader.mitpress.mit.edu

Post reply on HN
Print view View original

51–60 of 71 posts

Re: To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

#51

Post by arkades »

claudiawerner wrote: post #50
RenThraysk wrote: post #49

Earlier quoted context omitted.

It did prove that the only people capable of pointing out there were problems with the hoax papers are people generally critical of the social sciences. That alone demonstrates a problem. That's why the experiment ended early. PZ Meyers is a doxing buffoon.

How is that true when a good proportion of their papers were rejected, and they kept trying in other journals until they were accepted? What about the fact that several reviewers wrote critical responses and suggestions for improvement? Why can't what you said also be said about hoaxes in the "hard sciences" like this[0]? HN user voidhorse has a good comment about their tactics[1] (quote): > Sure, the criticism this…

Because the Medknow journals are notorious scam journals that will publish absolutely anything at all, without even a semblance of peer review?

There's a difference between non-prestige journal and vanity press.

Re: To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

#52

Post by RenThraysk »

claudiawerner wrote: post #50
RenThraysk wrote: post #49

Earlier quoted context omitted.

It did prove that the only people capable of pointing out there were problems with the hoax papers are people generally critical of the social sciences. That alone demonstrates a problem. That's why the experiment ended early. PZ Meyers is a doxing buffoon.

How is that true when a good proportion of their papers were rejected, and they kept trying in other journals until they were accepted? What about the fact that several reviewers wrote critical responses and suggestions for improvement? Why can't what you said also be said about hoaxes in the "hard sciences" like this[0]? HN user voidhorse has a good comment about their tactics[1] (quote): > Sure, the criticism this…

All papers get critical responses and suggestions for improvement. That proves nothing other than the submitted papers were considered to have some merit, and still possible inclusion in the various issues.

The problem is people (a journalist and RealPeerReview on twitter initially) identified quite a few of the hoax papers. And that does suggest there is a bubble within the field that has departed from usefulness and reality.

There is a problem within social sciences. Whatever is happening in hard science journals doesn't change that fact.

Re: To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

#53

Post by claudiawerner »

RenThraysk wrote: post #52
claudiawerner wrote: post #50

Earlier quoted context omitted.

How is that true when a good proportion of their papers were rejected, and they kept trying in other journals until they were accepted? What about the fact that several reviewers wrote critical responses and suggestions for improvement? Why can't what you said also be said about hoaxes in the "hard sciences" like this[0]? HN user voidhorse has a good comment about their tactics[1] (quote): > Sure, the criticism this…

All papers get critical responses and suggestions for improvement. That proves nothing other than the submitted papers were considered to have some merit, and still possible inclusion in the various issues. The problem is people (a journalist and RealPeerReview on twitter initially) identified quite a few of the hoax papers. And that does suggest there is a bubble within the field that has departed from usefulness an…

My claim was never that social science is immune to the replication crisis or that it could be more accepting of outside criticism; my claim was that singling out "grievance studies" (what the authors take to be sub-branches of critical theory) using poor research methods to make a name for oneself, childish tactics, not getting IRB approval isn't the right way to go about solving such a problem, and that using this as an example doesn't stand up to the large number of hoaxes/stings in hard sciences (see the first couple of links in my original comment). What suggests that the way to go about addressing this bubble (which can and does exist to at least some degree) is to publish more junk papers?

For a critical review of the papers' content, there's reason to believe that the hoax articles' premises may not be totally without merit[0].

"When Boghossian et al. describe their papers to us, the public, they do not explain what their bad arguments are, they only describe the “absurd” conclusions of the papers. So if the hoax is all about peer reviewers accepting bad arguments, then Boghossian et al. are failing to present the proper evidence, and propagating confusion about their own hoax."

[0] https://thingofthings.wordpress.com/2018/10/10/on-sokal-squa...

Re: To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

#54

Post by RenThraysk »

claudiawerner wrote: post #53
RenThraysk wrote: post #52

Earlier quoted context omitted.

All papers get critical responses and suggestions for improvement. That proves nothing other than the submitted papers were considered to have some merit, and still possible inclusion in the various issues. The problem is people (a journalist and RealPeerReview on twitter initially) identified quite a few of the hoax papers. And that does suggest there is a bubble within the field that has departed from usefulness an…

My claim was never that social science is immune to the replication crisis or that it could be more accepting of outside criticism; my claim was that singling out "grievance studies" (what the authors take to be sub-branches of critical theory) using poor research methods to make a name for oneself, childish tactics, not getting IRB approval isn't the right way to go about solving such a problem, and that using this…

I think you are arguing from bad faith. Social Sciences has problems and it seems unwilling to do better.

You had to be shamed publicly and widely. People have a right to know social sciences are this broken.

Re: To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

#55

Post by claudiawerner »

RenThraysk wrote: post #54
claudiawerner wrote: post #53

Earlier quoted context omitted.

My claim was never that social science is immune to the replication crisis or that it could be more accepting of outside criticism; my claim was that singling out "grievance studies" (what the authors take to be sub-branches of critical theory) using poor research methods to make a name for oneself, childish tactics, not getting IRB approval isn't the right way to go about solving such a problem, and that using this…

I think you are arguing from bad faith. Social Sciences has problems and it seems unwilling to do better. You had to be shamed publicly and widely. People have a right to know social sciences are this broken.

Perhaps, then, the authors should have published a critical review of existing social science literature and discussed flaws in methodology. That would be academically and ethically honest, and probably prompt a discussion and response from inside the field itself. Instead, they ran with their article to online news sites. Why would they do that first rather than trying to engage the academia themselves? Why did the researchers need to be shamed? See the quote I provided in my original comment by one of the reviewers who was fooled. She was personally embarrassed and thought the paper had come from a new entrant in the field, so she was charitable. Why is it acceptable that making people feel that way is the first course of action? It is not at all in the scientific spirit.

People have a right to know, of course - but so do researchers. Researchers in these fields have a right to know exactly what Boghossian and friends thought was wrong with their methodologies and research topics. Boghossian decided that was beneath him.

Re: To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

#56

Post by hn_throwaway_99 »

AlotOfReading wrote: post #27
choeger wrote: post #22

Earlier quoted context omitted.

> The results couldn't be related to the real world, and no one was really able to get over the inherent [1] influence of the author in the data. Wait. Do I understand you correctly that an objective approach failed and your conclusion is to be more subjective?

The "objective" approach was never actually objective. It had just as many biases and issues as everything after it, but everyone sort of pretended it didn't. The change was recognizing that the same data wasn't objective and that researchers couldn't make it so. Instead, theory shifted to analyzing those inherent biases and studying how they affected research.

> The "objective" approach was never actually objective. It had just as many biases and issues as everything after it, but everyone sort of pretended it didn't.

On a related note, I often see similar issues with the A/B testing programs in many startups and large companies. It's not that the objective approach with A/B testing is in any way wrong, it's that the constraints of the tests and the business environment mean it often isn't testing what you think it's testing.

For example, I've seen more than one A/B test where the business goal was to reduce the number of users taking some action (e.g. calling customer service, for example). So instead of fixing the underlying problem, or making online help easier or better to use, the A/B test basically just tests making the customer service link as hard to find/obscure as possible. Run the test for a few weeks, and voila, see how it's a big A/B test winner because support calls are way down!

Not all examples are egregious as that, but fundamentally virtually all of the A/B tests I've seen are limited to a relatively short time period, but aren't really able to test the longer term, follow on effects to brand reputation or customer sentiment.

Re: To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

#57

Post by lokisotube »

hliyan wrote: post #23

I've always found it uneasy that much of the data in social science studies comprise 5-point scale answers by volunteers to questionnaires, or observations of microcosmic situations that do not really represent reality (e.g. attempting to prove a hypothesis about generosity by giving participants $20 each and observing how they spend/gift it under a specially designed situation). I wonder whether questionnaires shoul…

Some questions to ponder: how is a record of behavior (the focus of behavioral science) any less objective than heart rate, pupil dilation, etc? Also, is heart rate a measure of psychological state really? Hormone levels are a mess as a measure of psychobehavioral state. EEGs are useful but are opaque in terms of underlying mechanisms.

People do use all the things you mention, and they're certainly useful, but also have limitations.

If you want to know how someone feels about liberal politics, for example (I'll let you pick the US or Britain), it's far easier and more direct to ask them than to try to infer it from heart rate responses etc.

I'm not saying those other things are useless, only that there's a reason Likert scales continue to be used so much.

Some food for thought from the other side of the coin:

https://aeon.co/essays/the-blind-spot-of-science-is-the-negl...

In any event, my complaint about the piece is that it picks on the social sciences when biomedicine is ripe with corruption and irreproducibility itself. There's a kind of bullying that occurs with this; biomedicine is rife with problems so it takes them out on a scapegoat. (The social sciences do have many problems, but many of them apply equally well to other fields.)

Re: To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

#58

Post by wmil »

adamsea wrote: post #6
Mountain_Skies wrote: post #5

The social sciences have gained an incredible amount of control over academia. Why would they allow the current system, which works very well for them, to be "fixed"?

Hmmm I do not know about that, the test real, meaningful control would be who gets the funding, no? Social sciences is nowhere near the top of this list. https://nsf.gov/statistics/2018/nsb20181/report/sections/aca... Overall my understanding is that it is harder to be an academic in almost any field these days, but especially the humanities - social sciences are an in-between zone, but I doubt the universities cutti…

> Hmmm I do not know about that, the test real, meaningful control would be who gets the funding, no?

Well no. That's arguably showing what politicians perceive the value of each department to be. But it doesn't say anything about power on campus.

The test of control should probably be what the undergrad degrees of the administrators and deans are.

Re: To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

#59

Post by naasking »

AlotOfReading wrote: post #39
malvosenior wrote: post #28

Earlier quoted context omitted.

If your data is not good enough, get better data. Don't abandon objectivity. I think this is why a lot of people don't take the social sciences seriously, there's an extreme resistance to just doing objective, data driven research (and the lack of reproducibility that comes with that).

Again, the issue isn't one of methodology or technology. It's inherent to the research process. Let's do a thought experiment and imagine two identical research projects done in parallel universes. Everything is the same between these two universes except that the researchers are different people. In some fields, you'd probably expect the results to be the same. The researcher shouldn't affect objective results after…

> The researcher shouldn't affect objective results after all. In a social science setting, the results may not be the same even if both researchers used exactly the same research design on exactly the same subjects at exactly the same point in their lives.

This is true of physics as well. Measurements in quantum experiments are contextual, in that the experimental setup interferes with system being measured. Measurements conform to statistical distributions. The difference is that physicists took the time to quantify how measurements influence the system, thus generating a practical theory. It's not clear to me that something similar couldn't be done here, even if human systems are in a sense, messier.

In fact, it's already been done to some extent because we clearly know how the phrasing of some questions influence the answers. We also know from tragedies of justice [1] how important this subject is.

Replication and a culture that encourages publishing negative results should be prerequisites for any field to qualify as a science.

[1] https://en.m.wikipedia.org/wiki/Day-care_sex-abuse_hysteria

Re: To Fix the Social Sciences, Look to the “Dark Ages” of Medicine

#60

Post by specialist »

ralusek wrote: post #15

For those interested in this, take a look at the work regarding "grievance studies" by Peter Boghossian, James Lindsay, and Helen Pluckrose. They made intentionally unscientific studies with the intent of publication, and actually got themselves published on a few of their works. The objective was to demonstrate that not only does academia in the social sciences operate according to a completely different standard th…

Their "study" reminds me of Penn & Teller getting people to sign a petition banning dihydrogen monoxide (aka water).

https://www.youtube.com/watch?v=yi3erdgVVTw

Just another form of social engineering. Haha, we compiled footage from a bunch of randos to show how stupid people. Haha, aren't we smart!

It's the academic equivalent of tricking someone into walking into a punch.

Post reply on HN