Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Elaborate Academic-Journal Hoax

chronicle.com

Post reply on HN
Print view View original

101–110 of 165 posts

Re: An Elaborate Academic-Journal Hoax

#101

Post by UncleMeat »

fenomas wrote: post #76
tptacek wrote: post #63

Discussed earlier on HN: https://news.ycombinator.com/item?id=18127811 The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals. They submitted multiple stories to some journals (notably, to Hypatia , which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was what made it in, rather than a much more banal paper. Th…

It's hard to tell specifically how their methodology went, but this bit of the author's write-up jumps out: > As for our performance, 80% of our papers overall went to full peer review, which keeps with the standard 10-20% of papers that are “desk rejected” without review at major journals across the field. We improved this ratio from 0% at first to 94.4% after a few months of experimenting with much more hoaxish pap…

The fact that they list 94.4% also gives up a hint. They iterated a lot. To get numbers that precise you need to submit a tremendous number of articles.

Give somebody a year and piles of submissions and of course they will sneak some things through.

Re: An Elaborate Academic-Journal Hoax

#103

Post by UncleMeat »

malvosenior wrote: post #98
UncleMeat wrote: post #78

Earlier quoted context omitted.

Peer Review is not set up to detect fraudulent research and attempts to do so are fraught with risk, since established researchers often have a hard time believing in results from new paradigms. What the heck is an author supposed to do if a reviewer says "I think these numbers are made up"? The purpose of peer review is to decide whether a paper is useful to the community, whether the presentation is good enough tha…

> The purpose of peer review is to decide whether a paper is useful to the community, whether the presentation is good enough that the community can precisely understand what happened, and whether it has methodological issues that the authors may have missed. It clearly failed to do that in this case.

Ok? That somebody with a year of free time and a pile of fraudulent attempts can get through sometimes is not exactly a surprise.

A system that prevents what the authors did here would necessarily reject a huge amount of useful research.

Re: An Elaborate Academic-Journal Hoax

#104

Post by UncleMeat »

malvosenior wrote: post #99
UncleMeat wrote: post #79

Earlier quoted context omitted.

To a reviewer with three hours to spare, fake research is indistinguishable from good research. Unless you want to give me months to review a paper, I will not be able to detect fraud. Instead I rely on the fact that the community will attempt to integrate these ideas after publication and arrive at the conclusion that the paper was garbage.

If you only have three hours, can't detect fraud, can't tell if the the paper is even real. Then what's the point of even doing the peer review? Again, peer review offered no value at all in this case.

You can certainly come to that conclusion if you'd like. Arxiv exists. But "peer review isn't especially usrful" and "these entire fields are nonsense" are very different conclusions.

Re: An Elaborate Academic-Journal Hoax

#105

Post by ArchTypical »

AnimalMuppet wrote: post #23
ArchTypical wrote: post #9

Earlier quoted context omitted.

It wasn't a hoax. It's a confounding detail that a meta-commentary article is employing.

Sorry, I can't parse that at all. Could you be a bit clearer on what you're trying to say here?

The article is attempting to explain the events, by characterizing them as a type of deception when it shows the exact opposite (it's not a hoax that papers are accepted on political aim, rather than evidence). It's a small detail, but telling as to the intent and voracity without knowing most the story.

Re: An Elaborate Academic-Journal Hoax

#106

Post by [deleted] »

Sniffnoy wrote: post #42

> Karl Steel, an associate professor of English at Brooklyn College and the Graduate Center of the City University of New York, called the trio’s work "simply not rigorous research" and described three objections to it. It is too narrow in disciplinary scope, he said. It focuses on exposing weaknesses in gender and ethnic studies, conspicuously ideological fields, when that effort would be better spent looking at mor…

[deleted]

Re: An Elaborate Academic-Journal Hoax

#107

Post by YeGoblynQueenne »

Sniffnoy wrote: post #42

> Karl Steel, an associate professor of English at Brooklyn College and the Graduate Center of the City University of New York, called the trio’s work "simply not rigorous research" and described three objections to it. It is too narrow in disciplinary scope, he said. It focuses on exposing weaknesses in gender and ethnic studies, conspicuously ideological fields, when that effort would be better spent looking at mor…

>> The whole point here is that you have these "conspicuously ideological" fields or subfields, which are not doing real scholarship, not interested in truth, but which rather are pushing a specific position and attempting to label as evil and shout down anyone who disagrees;

Yeah, look- it's not that simple. In academia, we have to allow discussion and debate to be carried out freely, but obviously it doesn't make sense to dictate the subject of a debate and call it "free"! And the only people who are qualified to choose the subject of the debate are academics themselves- because nobody else has the expertise to make that decision.

So if academics choose to debate identity politics and "Theory"- well, that's what they get to debate. If the debate is done in a rotten manner that is not conducive to good scholarship, now that's another matter, but if the question is if those fields should exist, the answer is yes, and the reason is that they do (it's a tautology, OK?).

It's a feature, not a bug. It's a bit like in a democracy you must allow everyone their political opinion, even those people who are against democracy itself? In academia, we must allow all academics the ability to investigate the subjects of their interest, else we don't have academia, we just have a closed club of people who agree with each other- so, just like the fields targeted by the hoax.

Re: An Elaborate Academic-Journal Hoax

#108

Post by YeGoblynQueenne »

darawk wrote: post #81
tptacek wrote: post #63

Discussed earlier on HN: https://news.ycombinator.com/item?id=18127811 The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals. They submitted multiple stories to some journals (notably, to Hypatia , which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was what made it in, rather than a much more banal paper. Th…

> The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals. What's your evidence for that? A number of their papers got accepted into decently high impact journals, including Hypatia. > They submitted multiple stories to some journals (notably, to Hypatia, which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was…

>> I would be extremely surprised if you could pass of work of such low quality in journals of comparable impact factor in [math, computer science, physics and economics].

What about medicine, genetics and neuroscience? These are the fields investigated by John Ioannidis, whom you've probably heard of [1]. He's the guy that published that paper titled "Why most published research findings are false" [2]. From the abstract of which, I quote:

Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.

Note that that's pretty much the claim of the hoax- that the hoaxed fields are dominated by bias and the majority of their research claims are absurd (so, false, I guess).

If that's true of medicine and genetics, it's true of most of the hard sciences, and no doubt, also CS and economics. So there is no good reason to single out gender studies or feminist philosophy for this kind of fault, or in any case, the hoaxers would have made a much more meaningful contribution if they had hoaxed a maths or physics journal, rather than the ones they did.

Btw- in maths, a recent HN article discussed the Mochizuki proof of the ABC conjecture, that his fellow maths experts consider to be very badly mistaken (if I got that right) [3].

_______________________

[1] https://en.wikipedia.org/wiki/John_Ioannidis#Research_findin...

[2] https://journals.plos.org/plosmedicine/article?id=10.1371/jo...

[3] https://www.quantamagazine.org/titans-of-mathematics-clash-o...

Re: An Elaborate Academic-Journal Hoax

#109

Post by darawk »

YeGoblynQueenne wrote: post #108
darawk wrote: post #81

Earlier quoted context omitted.

> The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals. What's your evidence for that? A number of their papers got accepted into decently high impact journals, including Hypatia. > They submitted multiple stories to some journals (notably, to Hypatia, which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was…

>> I would be extremely surprised if you could pass of work of such low quality in journals of comparable impact factor in [math, computer science, physics and economics]. What about medicine, genetics and neuroscience? These are the fields investigated by John Ioannidis, whom you've probably heard of [1]. He's the guy that published that paper titled "Why most published research findings are false" [2]. From the abs…

I am familiar with all of the situations you've cited, and they are nothing like this one. You're really misunderstanding Ionnandis' work if you think it is relevant to this. He's making arguments about bad statistics, which is super important and valuable, but this has almost nothing to do with that.

Again, the Mochizuiki situation is quite different. You're really not understanding these things if you think they're comparable. Mochizuiki's work is legitimate and real and serious. Whether or not it is correct is difficult to ascertain, because of its genuine complexity.

The accusation being made here is that these fields are incapable of separating absolute nonsense from genuine content. Which means that their work is indistinguishable from nonsense and absurdity. The work that Ioannidis did demonstrates that people acting within a good framework are producing work with a higher error rate than you might expect given the claims they make. That work is extremely important. But it doesn't invalidate the field of economics, or medicine. This work attempts to invalidate the entire edifice that these disciplines are built upon. It is extremely different and not comparable at all.

Re: An Elaborate Academic-Journal Hoax

#110

Post by darawk »

fenomas wrote: post #93
darawk wrote: post #81

Earlier quoted context omitted.

> The hoaxers writeup is, I think, pretty misleading. Their work tended to get rejected from high-impact journals. What's your evidence for that? A number of their papers got accepted into decently high impact journals, including Hypatia. > They submitted multiple stories to some journals (notably, to Hypatia, which they brag about getting accepted in) and leave a clear impression that their most ridiculous work was…

> You seem to think that the papers that got accepted were 'banal', but that doesn't change the fact that they were accepted. The banality of the papers is the central weakness of the whole thing. The whole point of Sokal- or SCIgen-style hoaxes is that the papers were gibberish - so when a journal publishes one, the author can legitimately say "Hey look, this journal published gibberish!". The authors here can't say…

> The banality of the papers is the central weakness of the whole thing. The whole point of Sokal- or SCIgen-style hoaxes is that the papers were gibberish - so when a journal publishes one, the author can legitimately say "Hey look, this journal published gibberish!".

My point is that most of them were not banal. In that line, I was responding to an isolated point of tptaceks. If all the papers were banal, i'd agree. But they were not.

Post reply on HN