Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Elaborate Academic-Journal Hoax

chronicle.com

Post reply on HN
Print view View original

51–60 of 165 posts

Re: An Elaborate Academic-Journal Hoax

#51

Post by mooman219 »

davemp wrote: post #16
eindiran wrote: post #10

Earlier quoted context omitted.

That's what stuck out to me as well. This quote: "I am so utterly unimpressed," wrote Jacob T. Levy, a political theorist at McGill University, "by the fact that an enterprise that relies on a widespread presumption of not-fraud can be fooled some of the time by three people with Ph.D.s who spend 10 months deliberately trying to defraud it." Isn't the ability to separate out the quality work from the garbage the poin…

> an enterprise that relies on a widespread presumption of not-fraud This is section really gets me. How in the world is science supposed to rely on “not-fraud”? Sure if you fake your data you might dupe a diligent reviewer, but this is why results are supposed to be reproduced. Anything that relies on “not-fraud” seems inherently unscientific and straight up naive.

Exactly this.

If my banking API relies on user's not trying to commit fraud instead of proper validation, then I have a very insecure API. Making a statement that your system works as long as no one tries to break it sounds almost satirical.

Re: An Elaborate Academic-Journal Hoax

#52

Post by entwife »

Journals are for reports from scientists and academics. Would they be more useful if there was some additional review of the authors reputation, and the reliability of the particular report? Upvoting or public signing of the documents, or public signed comments. An expert in a field is more familiar with the reputation of their colleagues than outsiders.

Re: An Elaborate Academic-Journal Hoax

#53

Post by brianberns »

> I don't want to know whether social constructionism is considered "problematic", I want to know whether it is true.

Not all legitimate scholarly work is based on a search for scientific truth. E.g. Interpretations/analysis of Shakespeare's work.

Re: An Elaborate Academic-Journal Hoax

#54

Post by edouard-harris »

The authors state that this was a long term research project cut short. Which suggests the question:

If you played this game long enough, undetected, could you get real scholars to cite your fake research?

Future work, perhaps.

(I don't have any direct evidence of this, but I strongly suspect this is an hypothesis that the authors would have wanted to test.)

Re: An Elaborate Academic-Journal Hoax

#55

Post by adrianratnapala »

UncleMeat wrote: post #48
eindiran wrote: post #10

Earlier quoted context omitted.

That's what stuck out to me as well. This quote: "I am so utterly unimpressed," wrote Jacob T. Levy, a political theorist at McGill University, "by the fact that an enterprise that relies on a widespread presumption of not-fraud can be fooled some of the time by three people with Ph.D.s who spend 10 months deliberately trying to defraud it." Isn't the ability to separate out the quality work from the garbage the poin…

Not at all. Having reviewed papers, my goal is to make the underlying experiment as clear as possible for readers when the paper is eventually published. I care about the methods being clear and the analysis being sound. I am absolutely not looking for "I think the authors are lying" warnings because this way lies madness. If we expect reviewers to reject papers they find weird or suspicious then a lot of excellent s…

That's because reviewers of experimental papers are not expected to reproduce experiments. In theoretical physics or mathematics a reviewer absolutely should follow the proofs and check they are sound.

The humanities (as opposed to the social sciences) are theory fields, they don't have experiment but they advance knowledge by good arguments that require real scholarship to support even if they don't have crispness of proofs. Someone reviewing it should go "Yep, that paper make a real claim and argues for it well enough that convincing counter-argument would need to be a significant work of scholarship in itself".

If a field of study produces a conversation made up of such arguments and counter-arguments, then it stands some chance of being growing repository of truth. Whereas if the field can't distinguish between it's own chatter and a hoax guided by mood affiliation, then the field is likely to be just chatter.

Re: An Elaborate Academic-Journal Hoax

#56

Post by TheCoelacanth »

davemp wrote: post #16
eindiran wrote: post #10

Earlier quoted context omitted.

That's what stuck out to me as well. This quote: "I am so utterly unimpressed," wrote Jacob T. Levy, a political theorist at McGill University, "by the fact that an enterprise that relies on a widespread presumption of not-fraud can be fooled some of the time by three people with Ph.D.s who spend 10 months deliberately trying to defraud it." Isn't the ability to separate out the quality work from the garbage the poin…

> an enterprise that relies on a widespread presumption of not-fraud This is section really gets me. How in the world is science supposed to rely on “not-fraud”? Sure if you fake your data you might dupe a diligent reviewer, but this is why results are supposed to be reproduced. Anything that relies on “not-fraud” seems inherently unscientific and straight up naive.

Reproduction of results happens after publication, though. Simply getting published does not demonstrate that their ideas were accepted as truthful, you should look at things like whether their work was cited by other work.

Re: An Elaborate Academic-Journal Hoax

#57

Post by TheCoelacanth »

adrianratnapala wrote: post #55
UncleMeat wrote: post #48

Earlier quoted context omitted.

Not at all. Having reviewed papers, my goal is to make the underlying experiment as clear as possible for readers when the paper is eventually published. I care about the methods being clear and the analysis being sound. I am absolutely not looking for "I think the authors are lying" warnings because this way lies madness. If we expect reviewers to reject papers they find weird or suspicious then a lot of excellent s…

That's because reviewers of experimental papers are not expected to reproduce experiments. In theoretical physics or mathematics a reviewer absolutely should follow the proofs and check they are sound. The humanities (as opposed to the social sciences) are theory fields, they don't have experiment but they advance knowledge by good arguments that require real scholarship to support even if they don't have crispness o…

If the field doesn't rely on experimentation, why did their papers include fake experimental data (e.g. one of the papers contained a bunch of fake data about their observations of dog genitalia)?

Re: An Elaborate Academic-Journal Hoax

#58

Post by TheCoelacanth »

blastforses wrote: post #33
lukev wrote: post #2

Representative quote: > "I am so utterly unimpressed," wrote Jacob T. Levy, a political theorist at McGill University, "by the fact that an enterprise that relies on a widespread presumption of not-fraud can be fooled some of the time by three people with Ph.D.s who spend 10 months deliberately trying to defraud it."

The whole point of a peer review system is to detect fraud. If these bullshit papers got published, who knows how many of the other papers are of equal quality.

The point of peer review is to stop bad research, not fake research. The reviewers are supposed to make sure that the methods used to acquire the data are clearly documented and that the conclusions drawn from the data follow from the data that was collected.

They can't verify that the data collected actually was collected without literally watching over the shoulder of the researcher while the researcher was collecting it, which is obviously not feasible. A researcher actively trying to deceive the reviewer with fake data but with a good method for collecting that data and analysis of the data that would be correct if the data were real is not something that should be caught by a reviewer. That should be caught by someone reproducing the research after publication.

Re: An Elaborate Academic-Journal Hoax

#60

Post by amw »

PinkMilkshake wrote: post #59

I cant help but give a Gilfoyle style heh... heh... heh.... What I hope has happened is the 20 revealed papers are not actually all of them and that they do a Wikileaks style drip release, wait on a response, then hit us with more much later.

If they had those results, they would have put them front and center. Just like if the TSA had ever caught a terrorist, it would be on the 24-hour networks for days.
Post reply on HN