I think scientific journals and other infrastructures that have the potential to greatly impact society need to put defense mechanisms in place for bad actors going forward.
Or we should stop taking (everything) published by as gospel.
[deleted]
Another scam paper published in a “scientific” journal
11–20 of 50 posts
Re: Another scam paper published in a “scientific” journal
#12As a lay-person who reads scientific papers sometimes because they’re referenced on HN, how do I validate the credibility of a journal? Does a directory of journals exist with scoring, or is there a strategy I can use when evaluating a paper or journal to determine credibility?
There are impact factors and top lists, and you can check out the Wikipedia page of the journal if it exists. You can google what are the "top journals for ". You can check the publisher. IEEE and Springer for example tend to be genuine.
But you still have to look at the individual paper and can't believe it just based on a single article. Research articles are for sharing results among experts (and for advancing the careers of researchers), they are not aimed at laypeople. It's easy to misinterpret them if you lack the background knowledge.
Even in non-predatory journals many results fail to replicate and are produced due to a publish or perish pressure. You're better off learning from textbooks so you get info that has been verified, digested and distilled and represent consensus. Cutting edge research proposes new ideas by one group of authors, it's not a consensus yet.
Re: Another scam paper published in a “scientific” journal
#13As a lay-person who reads scientific papers sometimes because they’re referenced on HN, how do I validate the credibility of a journal? Does a directory of journals exist with scoring, or is there a strategy I can use when evaluating a paper or journal to determine credibility?
One possible way is looking at the list of editors of the Journal, and seeing if the researchers in question have back-mentioned that Journal on their CV (or personal homepage). The fact that researchers/peers in a field are willing to openly connect their prestige to those of a journal's, is usually an indicator that there is at least something to the journal.
Most if not all scammy journals will fail this test.. but it is also quite labor intensive.
Re: Another scam paper published in a “scientific” journal
#14I think scientific journals and other infrastructures that have the potential to greatly impact society need to put defense mechanisms in place for bad actors going forward.
Why would they? These journals make money from authors paying for a publication. Any review costs cut into their margins.
Also, I don’t think this journal has even the potential to greatly impact society. It isn’t intended to be read.
Re: Another scam paper published in a “scientific” journal
#15Ok, so here's what you do next: make a list of all of the authors published in those journals and track their careers.
Anyone want to bet the results will be entertaining?
Re: Another scam paper published in a “scientific” journal
#16I would love to read a parody scientific journal, we may even stumble on some hidden truths.
Re: Another scam paper published in a “scientific” journal
#17As a lay-person who reads scientific papers sometimes because they’re referenced on HN, how do I validate the credibility of a journal? Does a directory of journals exist with scoring, or is there a strategy I can use when evaluating a paper or journal to determine credibility?
There are a number of metrics[1] for journal ranking. The journal in question[2] claims to have (or have had) an impact factor[3] of 0.593 (2017-18); "the ratio between the number of citations received in that year [...] and the total number of "citable items" published in that journal[...]". That's not a very good score. Nature has an impact factor of ~42 (2019), or to pick another example, because I recently referred to it here on HN, Eurosurveillance has an impact factor of ~6.
I don't think blindly following these metrics is a good idea, but it's not as a bad first approximation.
[1] https://en.wikipedia.org/wiki/Journal_ranking
Re: Another scam paper published in a “scientific” journal
#18Ok, so here's what you do next: make a list of all of the authors published in those journals and track their careers. Anyone want to bet the results will be entertaining?
Unfortunately, the truth is much more unfunny than you think. In the past, you could get tenure at community or regional colleges just by teaching. If you were interested you might from time to time publish an article in the institution's journal or maybe even in a real journal - as a botanist or astronomer you can do citizen science without much equipment. That's how it ought to be.
But nowadays, administration has taken over, and everyone gets to fulfill the publication metric, or there won't be tenure. So you have predatory journals (I prefer the term "journal of convenience") that fulfill a genuine need. The only one who is worse off is the taxpayer or tuition-paying student because they get to pay the page charges. That is a problem, but everyone reacts to incentives.
There are certain parallels to zero-tolerance policies at schools - in the past the popular kids got off lightly, consequentely because of real or perceived corruption the popular kids now get away with no punishment at all while everyone else has a 5-ton weight dropped on them, all in the name of fairness.
Re: Another scam paper published in a “scientific” journal
#19As a lay-person who reads scientific papers sometimes because they’re referenced on HN, how do I validate the credibility of a journal? Does a directory of journals exist with scoring, or is there a strategy I can use when evaluating a paper or journal to determine credibility?
A really simple good-enough approach is to go to Google scholar, type in the title, and count how many citations it gets. Adjust for years since publication. Many citations: probably important. Few citations: less important. No citations, more than a year old: probably not much good.
It isn't perfect. Different disciplines tend to cite at different rates; and you may believe that some disciplines are just cabals of bullshitters citing each other. I couldn't possibly comment.
Re: Another scam paper published in a “scientific” journal
#20As a lay-person who reads scientific papers sometimes because they’re referenced on HN, how do I validate the credibility of a journal? Does a directory of journals exist with scoring, or is there a strategy I can use when evaluating a paper or journal to determine credibility?
One doesn't validate the credibility of a journal; one validates the credibility of the methodology in the research. And it then turns out that most methodology even in reputable journals are rather wanting with many objections that can be leveled against it. A large amount of scientific research can't even be reproduced, and of much that can, even though the cold data can be reproduced, the conclusions that follow f…
It absolutely makes sense to simply ignore articles based on the credibility of the journal, it's an effective and cheap first filter (and you absolutely need filters) - there are many predatory journals (like this one) which will publish anything that's paid for, they probably even outnumber "real" journals, and it makes all sense to automatically discard them without reading.
There is a lot of noise already in "proper" journals - but in the predatory journals, the signal-to-noise ratio is so extremely low, it's not worth looking into the credibility of the methodology of the paper because that's far more time and effort that the paper deserves. If it was any good, it would have been published in a better venue. If it could pass peer review, it would have been published in a venue that actually does peer review as opposed to these (many) predatory journals who just claim to do so. The authors have strong practical incentives to not publish it there if they can avoid to, and the fact that they chose to do so anyway indicates that no respectable place would publish it.
Because of that, if a paper is published in a place like this, is a completely reasonable prior to presume that overwhelmingly likely the paper is very bad, without even looking at the paper itself.