Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fake scientific papers are alarmingly common

science.org

Post reply on HN
Print view View original

41–50 of 105 posts

Re: Fake scientific papers are alarmingly common

#41

Post by ekianjo »

Turukawa wrote: post #21

The researchers in this paper use an astonishingly biased "fake paper detector", requiring only two conditions to be met for any paper to be considered "fake": 1. Use a non-institutional email address, or have a hospital affiliation, 2. Have no international co-authors. And they acknowledge 86% sensitivity and 44% specificity. It's a coin-toss which biases massively against research from outside the US and Western Eu…

> The researchers in this paper use an astonishingly biased "fake paper detector"

I havent looked at the details here, but if you make a prediction model and if that prediction model is robust enough to explain with great accuracy something with 2 or 3 variables, it's not going to be "biased", it's just going to be robust and right more often than not using only these few variables (as long as the training data was broad enough).

Re: Fake scientific papers are alarmingly common

#42

Post by newswasboring »

largepeepee wrote: post #28
rst wrote: post #10

The metrics used in this paper are... deeply flawed, to the point that the authors admit that they label nearly half of known good papers in a curated sample as "fake" -- and particularly likely to generate false positives for researchers whose institutions don't, say, run their own email systems (as is common in large chunks of the world). Here's a rundown of the flaws from an epidemiologist with a sideline in scien…

You know what's funny? Even if the numbers are hot garbage, they proved the point about how easy it is to publish fake science papers, since it got published. Kinda similar to those researchers years back who proved how easy it was to go into certain social science journals as long as you copied their ideology.

It's a medRxive preprint. It didn't get published anywhere. Science (the magazine) has lowered it's standards.

Re: Fake scientific papers are alarmingly common

#43

Post by Gareth321 »

alsodumb wrote: post #4

Unfortunately, it's an open secret that fake or low-effort almost useless papers are very common in every area of scientific research. Typically, it doesn't affect people working in that specific area - they develop/have a sixth sense to detect bullshit papers - it comes with experience but depends on several factors including the authors reputation, their institution (for the first screening), what journal/conferenc…

My favourite example is the grievance studies (https://en.wikipedia.org/wiki/Grievance_studies_affair). The authors published, among others, portions of directly copied Mein Kampf. In one "study" submitted, they claimed to have observed thousands of hours of dogs having sex in parks, to observe the "patriarchal" linked to "rape culture". The entire thing is a horrible indictment on the level of scrutiny undertaken in the various activist and social "science" oriented journals.

Re: Fake scientific papers are alarmingly common

#44

Post by tgv »

FabHK wrote: post #30
Turukawa wrote: post #21

The researchers in this paper use an astonishingly biased "fake paper detector", requiring only two conditions to be met for any paper to be considered "fake": 1. Use a non-institutional email address, or have a hospital affiliation, 2. Have no international co-authors. And they acknowledge 86% sensitivity and 44% specificity. It's a coin-toss which biases massively against research from outside the US and Western Eu…

No. They use 400 known fakes and 400 matched (presumed) non-fakes to estimate the sensitivity and specificity of their indicator, then apply that indicator to the full universe, then employ the estimated sensitivity and specificity to the obtained measurement to estimate the approximate actual rate of false papers. If you know the true prevalence of a disease in a population, and the sensitivity and specificity of yo…

Their baseline had better be representative.

Re: Fake scientific papers are alarmingly common

#45

Post by alsodumb »

version_five wrote: post #36
alsodumb wrote: post #4

Unfortunately, it's an open secret that fake or low-effort almost useless papers are very common in every area of scientific research. Typically, it doesn't affect people working in that specific area - they develop/have a sixth sense to detect bullshit papers - it comes with experience but depends on several factors including the authors reputation, their institution (for the first screening), what journal/conferenc…

Right- academic papers are written for academicians who don't have any issues separating good papers and journals from bad. The fact that many journals have set themselves up as or allowed themselves to become part of the tenure and promotion metrics game, is more of an issue with tenure and promotion. If the requirement for simple metrics dissapeared, the fake papers would go away on their own. In any event, it's no…

Yup, that’s sums up the incentive for publishing so many papers and get citations.

Some professor put it in a nice way - the current system motivates us to think of research in terms of LPUs - least publishing units. No matter how established your lab is, you’d try to publish as soon as possible, leading to a lot of papers with not a lot of contribution. If tenure committees and all other systems that gauge academicians require people to say present their only top 3 or 5 seminal papers, then people would try to put their best work out there without the constant pressure of always publishing - win win for everyone. Unfortunately, the ones with the power to make these changes are the ones gaining the most in the current system so it’s unlikely to happen.

Re: Fake scientific papers are alarmingly common

#46

Post by [deleted] »

alsodumb wrote: post #4

Unfortunately, it's an open secret that fake or low-effort almost useless papers are very common in every area of scientific research. Typically, it doesn't affect people working in that specific area - they develop/have a sixth sense to detect bullshit papers - it comes with experience but depends on several factors including the authors reputation, their institution (for the first screening), what journal/conferenc…

[deleted]

Re: Fake scientific papers are alarmingly common

#48

Post by cauch »

largepeepee wrote: post #28
rst wrote: post #10

The metrics used in this paper are... deeply flawed, to the point that the authors admit that they label nearly half of known good papers in a curated sample as "fake" -- and particularly likely to generate false positives for researchers whose institutions don't, say, run their own email systems (as is common in large chunks of the world). Here's a rundown of the flaws from an epidemiologist with a sideline in scien…

You know what's funny? Even if the numbers are hot garbage, they proved the point about how easy it is to publish fake science papers, since it got published. Kinda similar to those researchers years back who proved how easy it was to go into certain social science journals as long as you copied their ideology.

Well, there is a difference between "fake science" and "tried to do correct science but ending being wrong". If the second is "fake science", then basically all that Newton has ever produced is "fake science".

For the social science journals bit, are you thinking of the "grievance studies affair": https://en.wikipedia.org/wiki/Grievance_studies_affair ?

Ironically, this study has generated a lot of "fake news" on the field of social science. The conclusions of this study were widely spread mainly by people for ideological reason. When we look at the study in question, it's clear the conclusions are quite different than what the rumors say. For example, the same researchers tried such hoax before the ones they mention in their study, except that these hoaxes failed to be published, and they "forgot" to mention it. They did not have any control group, neither as "correct article" or "article defending the opposite ideology" (so, how can we conclude that the reason these bad articles were published were because of ideology if you don't know how many articles are published without being critically reviewed). They also count as valid a lot of journals that are pay-to-publish and not seriously used in the field. One of the author, ironically, ended up supporting platforms publishing conspiracy theories (and he was even banned from Twitter) (not that the study should be judged based on that, but it's a funny anecdote: the author who, according to some, had the courage to defend real science against bad woke ideology, who ends up demonstrating that he never cared about real science and is driven by ideology not science)

Re: Fake scientific papers are alarmingly common

#49

Post by DoreenMichele »

I've read other stuff related to this issue. It seems to me our current system exists in a social reality for which our metrics of authenticity were not designed and it harms both credentialization -- which is recognized as a problem -- and also serious science in ways that are not readily acknowledged as a problem.

Mendel, father of genetics, failed to become an accredited teacher. His work on genetics would likely get no recognition in this environment of credentialism is king.

Some guy who knows enough about genetics he created his own home pill to deliver genes into his gut to fix his lactose intolerance is being ignored by the world. Someone recently told me on HN that his video sounds like a scam video of a sort that is common (probably in a redacted comment).

I have a genetic disorder, which fails to pass the credentialism test. For that and other reasons, I didn't bother to say anything like "Sorry you don't know enough about genetics to follow it."

The individual wanted to know where the "studies" and "papers" were. And they likely don't exist and will never exist because there's no profit in it for someone else to try to build on his work.

I don't know how we fix this, but the world has changed and it's valuing the facade of scientific work more than actual scientific work and it makes me want to scream.

Post reply on HN