Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Another scam paper published in a “scientific” journal

whyevolutionistrue.com

Post reply on HN
Print view View original

21–30 of 50 posts

Re: Another scam paper published in a “scientific” journal

#21

Post by jonnycomputer »

morsch wrote: post #17
vegannet wrote: post #5

As a lay-person who reads scientific papers sometimes because they’re referenced on HN, how do I validate the credibility of a journal? Does a directory of journals exist with scoring, or is there a strategy I can use when evaluating a paper or journal to determine credibility?

There are a number of metrics[1] for journal ranking. The journal in question[2] claims to have (or have had) an impact factor[3] of 0.593 (2017-18); "the ratio between the number of citations received in that year [...] and the total number of "citable items" published in that journal[...]". That's not a very good score. Nature has an impact factor of ~42 (2019), or to pick another example, because I recently referr…

Impact factor works if you just to minimize the number of "bad" publications you look at. But if you rely on it too much, you are going to exclude lots of good publications too. Specialist journals tend to have lower impact factors, for one. Journal of Artificial Societies and Social Simulation is a good journal, but its impact factor has never been especially high.

Re: Another scam paper published in a “scientific” journal

#22

Post by chrisseaton »

HarryHirsch wrote: post #18
mcguire wrote: post #15

Ok, so here's what you do next: make a list of all of the authors published in those journals and track their careers. Anyone want to bet the results will be entertaining?

Unfortunately, the truth is much more unfunny than you think. In the past, you could get tenure at community or regional colleges just by teaching. If you were interested you might from time to time publish an article in the institution's journal or maybe even in a real journal - as a botanist or astronomer you can do citizen science without much equipment. That's how it ought to be. But nowadays, administration has…

I don't understand this claim - because publishing in a 'journal of convenience' won't work for tenure because they don't just count your publications they look at what impact your publications have.

Re: Another scam paper published in a “scientific” journal

#23

Post by nabla9 »

vegannet wrote: post #5

As a lay-person who reads scientific papers sometimes because they’re referenced on HN, how do I validate the credibility of a journal? Does a directory of journals exist with scoring, or is there a strategy I can use when evaluating a paper or journal to determine credibility?

When you see link to paper with interesting title posted to HN from nature.com domain, it's almost certainly from "Scientific Reports". It's not as bad as described here, but nobody publishes there if they have paper that is scientifically interesting. It's place to report what you did when nothing interesting came out of it.

Nature publishes many journals and "Nature" is the highest quality general science publication. "Scientific Reports" does just quick review for methodology.

Re: Another scam paper published in a “scientific” journal

#25

Post by Blikkentrekker »

PeterisP wrote: post #20
Blikkentrekker wrote: post #8

Earlier quoted context omitted.

One doesn't validate the credibility of a journal; one validates the credibility of the methodology in the research. And it then turns out that most methodology even in reputable journals are rather wanting with many objections that can be leveled against it. A large amount of scientific research can't even be reproduced, and of much that can, even though the cold data can be reproduced, the conclusions that follow f…

It absolutely makes sense to simply ignore articles based on the credibility of the journal, it's an effective and cheap first filter (and you absolutely need filters) - there are many predatory journals (like this one) which will publish anything that's paid for, they probably even outnumber "real" journals, and it makes all sense to automatically discard them without reading. There is a lot of noise already in "pro…

This is true; it works in one direction, but not in the other.

But how the post which I replied to was worded suggested that one can automatically trust “science”, provided that it be published in a credible journal.

I also find that very often in the most reputable journals is where the most sensational papers end up before any attempt at reproduction has been made simply because the data they measured was far from the null hypothesis, which may entirely be a statistical fluke and not hold up under a reproduction attempt.

It is really quite easy to obtain spectacular data as a fluke.

Re: Another scam paper published in a “scientific” journal

#26

Post by HarryHirsch »

chrisseaton wrote: post #22
HarryHirsch wrote: post #18

Earlier quoted context omitted.

Unfortunately, the truth is much more unfunny than you think. In the past, you could get tenure at community or regional colleges just by teaching. If you were interested you might from time to time publish an article in the institution's journal or maybe even in a real journal - as a botanist or astronomer you can do citizen science without much equipment. That's how it ought to be. But nowadays, administration has…

I don't understand this claim - because publishing in a 'journal of convenience' won't work for tenure because they don't just count your publications they look at what impact your publications have.

"College" has a wide range of meanings in the US - Cornell is a college, but so is Yoknapatawpha County Community College in Mississippi, where 20 % of students are functionally illiterate.

Of course they'll consider impact factor at Cornell, but at Yoknapatawpha they just count.

Re: Another scam paper published in a “scientific” journal

#27

Post by peter_retief »

teraflop wrote: post #24
peter_retief wrote: post #16

I would love to read a parody scientific journal, we may even stumble on some hidden truths.

Check out the Journal of Irreproducible Results and its competitor, the Annals of Improbable Research .

Thanks I will do so.

Re: Another scam paper published in a “scientific” journal

#29

Post by corty »

vegannet wrote: post #5

As a lay-person who reads scientific papers sometimes because they’re referenced on HN, how do I validate the credibility of a journal? Does a directory of journals exist with scoring, or is there a strategy I can use when evaluating a paper or journal to determine credibility?

Actually, asking for the credibility of a journal is the wrong question. The real one is about whether an article is credible. But the answer to that is disappointingly complicated.

Articles are peer-reviewed for their readability (for peers, not laymen), consistency (e.g. if there is a mathematical proof in there, the reviewer might check it if possible. if there is an experimental technique, the reviewer might check if that technique could produce the results claimed) and reproducability (is the experiment described in sufficient detail? does the software provided produce the expected results? do the provided numerical results fit the claims?). All those checks are what the reviewer can do in a few hours, days at most. That is what peer-review means.

All the things to check beyond those are left as an exercise to other scientists after publication. Reproduction is a big task, usually almost as big as the original paper. Proofs are often extremely hard to check, so a few days by any old reviewer won't cut it. So if you want to know if the results in a paper are true, watch literature for the following years and look at publications citing that paper and (dis-)agreeing with its results. Same for (sometimes) letters to the editor, retractions and "everybody-knows"-rumours.

More reputable journals tend to attract better-quality papers (according to the peer-review criteria outlined above). That tends to correlate with a higher probability of the paper being true. But it is not a very strong correlation, there is utter nonsense, groupthink and polished turds even in very high profile journals.

Re: Another scam paper published in a “scientific” journal

#30

Post by Blikkentrekker »

bonoboTP wrote: post #12
vegannet wrote: post #5

As a lay-person who reads scientific papers sometimes because they’re referenced on HN, how do I validate the credibility of a journal? Does a directory of journals exist with scoring, or is there a strategy I can use when evaluating a paper or journal to determine credibility?

There are impact factors and top lists, and you can check out the Wikipedia page of the journal if it exists. You can google what are the "top journals for ". You can check the publisher. IEEE and Springer for example tend to be genuine. But you still have to look at the individual paper and can't believe it just based on a single article. Research articles are for sharing results among experts (and for advancing the…

> You're better off learning from textbooks so you get info that has been verified, digested and distilled and represent consensus.

A very large body of consensus ended up in textbooks that had either never been attempted te be reproduced, or was attempted, much later, and couldn't be reproduced, yet continued to remain in textbooks.

The truth of the matter is that most scientific research will never see an attempt at replication, though the replication crisis has no doubt influenced this culture, but before it, as little as 0.16% of peer reviewed results were attempts at replication, and most were unsuccessful. — of course, the successful ones were not as easily published, which is probably why no one attempted it.

Post reply on HN