Live data from Hacker News

Government grant-funded research should not be published in for-profit journals

experimental-history.com

31–40 of 154 posts

Re: Government grant-funded research should not be published in for-profit journals

#31
post #10

We already have open-access publications: Just put it on arXiV. Most researchers I work with do this already. The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything. TPCs don't use it in their list of citations, neither do grant funding agencies or government institutions. The current academic enterprise relies heavily on third-party gatekeeping.…

> The first thing an academic does is check where a paper is published, before even reading it. It's a crutch

IMO, academics that do this are not very competent, because we have plenty of research suggesting that higher-profile journals are in fact less trustworthy in many ways, or that there is no correlation at all between reputation and quality (see my other post here in this thread).

Yes, some trash journals publish all trash, but, beyond that, competent researchers scan the abstract, look at sample sizes and basic stats, and if those check out, you skip to the methods and look for red flags there. Also, most early publications will be on an arXiv-like place anyway so you can't look to reputation yet.

Likewise, serious analytic reviews like meta-analyses don't factor in e.g. impact factor or paper citations, since that would be nonsense. They focus on methodology and stats.

I really think we ought to shame academics that are filtering papers based on journal alone, it is almost always the wrong way to make a quick judgement.

Re: Government grant-funded research should not be published in for-profit journals

#32
post #27

Earlier quoted context omitted.

Unfortunately I think charging money is a necessary signal that this particular gatekeeper is doing a good job. We should recognise that money is a necessary part of this process, else there is no gate to keep. But we shpuld reverse the economics by having people pay to get their stuff peer reviewed. Imagine if reviewing research papers was something you could get paid to do, the incentive then isnt to rubber stamp t…

> I think charging money is a necessary signal that this particular gatekeeper is doing a good job. I’ve never seen the slightest relationship between the charge to read a paper and the quality of review.

Because there isn't such a relation. It's a thing people believe when they don't have actual experience with peer review. If anything, predatory journals and low-quality pubs can charge more, since publication is more guaranteed (and researchers reaching for these pay-to-publish journals are more desperate).

Re: Government grant-funded research should not be published in for-profit journals

#33
Acknowledging, I am not a expert in this stuff, here is an idea: getting momentum for these sorts of things is so important, what is the journal that would be easiest to make a big example of, so that everyone understands that it is possible? Just completely mercilessly drive them out of business, and then hound their executives when they try to get other jobs. It appeals to peoples base instincts, but the last 10 years have shown those are pretty powerful. Then the movement which has formed around that can take down progressively bigger journals. Probably want a different organization building the alternative; the people with the personality to fight at the Vanguard of the revolution don’t tend to be great at building in the long-term.

Re: Government grant-funded research should not be published in for-profit journals

#34
Journals are not about providing access to science, much less public access.

Journals are an academic-career-advancement service. It therefore makes sense that they do not pay academics. You don't pay your customers.

That means they need to generate a secondary customer base elsewhere, who will pay. Those secondary customers happen to be the employers of the academics who are the primary customers. That socializes the cost of providing the service, since academics individually wouldn't be willing and able to pay.

Once journals have established a reputation, their policies and paywalls and fees are the result of trying to signal exclusivity and set an optimum market price.

Until the supply side of the research market largely agrees on a way to use open-access repositories like arXiv as a primary career-advancement signal, complaining about closed-access journals is tilting at windmills.

Changing the law to prevent journals from being able to copyright anything could potentially force the research industry to rapidly develop a new solution, but at the cost of short-term chaos and career instability for new academics.

Re: Government grant-funded research should not be published in for-profit journals

#35
> Robert Maxwell, one of the architects of the for-profit scientific publishing scheme. When he later went into debt, he plundered hundreds of millions of pounds from his employees’ pension funds. You may be familiar with his daughter and lieutenant Ghislaine Maxwell, who went on to have a successful career in child trafficking.

Wow! Surprised that hasn't been mentioned here already. Jumped out to me immediately as a morbidly curious bit of trivia.

Re: Government grant-funded research should not be published in for-profit journals

#36
I don’t understand why people care so much about the cost of journal subscriptions. If we add up all the revenue from all major scientific journal publishers, is that a big number in the context of the national economy? Or even compared to one major tech company?

I feel like this is one of those classic local minima where a community starving for resources fights vociferously amongst itself because they have internalized that they can’t win externally. From where I sit outside academia the problem with science seems obvious: there is not nearly enough money going into it.

I doubt bringing the heads of for-profit journals would change that under current national conditions in the U.S.

Re: Government grant-funded research should not be published in for-profit journals

#37

Earlier quoted context omitted.

> How does that have anything to do with peer review? I already addressed this. People know peer review can be bad, but some think "good journals" still do good peer review. This is not so clear. > In what world does the arxiv system moderate this discrepancy? Open systems allow the scientific community to figure out ways to properly assess research quality and value more cheaply, and without passing through (often a…

Your argument depends on worse peer review at top journals - but fundamentally, you fail to show how doing any peer review is strictly worse than doing no peer review. I understand that we want arxiv to exist, and it does, and it’s growing. That doesn’t mean we don’t want Nature or Science to triage the most compelling stories. Importantly, we can already begin the search for these ‘cheaper’ review strategies while n…

> Your argument depends on worse peer review at top journals - but fundamentally, you fail to show how doing any peer review is strictly worse than doing no peer review.

No, it doesn't. The argument is that peer review is incompetent gatekeeping in general, and so slows things down and makes thing expensive. Also, I am countering the argument "we need journals because journals do peer review" by arguing "peer review by journals isn't clearly actually good", I am not saying "peer review in general is unneeded", as I support review by the entire scientific community, rather than journal gatekeepers.

> you fail to show how doing any peer review is strictly worse than doing no peer review

I wasn't trying to show that. I have provided plenty of arguments to show why killing journal-based peer review could definitely speed things up and so potentially make things better. I want actual organic review by the community, not by tiny groups of gatekeepers.

Re: Government grant-funded research should not be published in for-profit journals

#38

The other factor preventing a fix is that people with no actual serious experience of academic publishing and peer review will defend these journals, because they still think that (journal-based) peer review acts like some kind of meaningful quality filter. But, it really doesn't. Because someone is surely going to try to defend journals via peer review in this thread, I want to provide a counter to the arguments tha…

> higher-profile journals in general have more retractions and research misconduct [1-2]

Given that reviews are not a mechanism to check for truth but soundness, the higher profile the thing I would imagine there would be more misconduct. I mean would one risk prison to steal 10$ or to steal 1 million $?

> lower research quality [3]

To cite exactly from your link "the evidence is mixed about whether they are strongly correlated with indicators of research quality.". I find saying "lower" a bit too strong given the original quote.

> in fact weaker statistical power and reliability [4]

For a specific field "cognitive neuroscience and psychology papers published recently"!

> statistical reliability even in high prestige journals is still extremely poor overall [5]

According to https://www.frontiersin.org/journals/human-neuroscience/arti... they kind of targeted bio/medical/psychology field for the analysis. Which seems to me very focused to be able to draw general conclusions.

> Also, making it through peer review is highly random and dependent on who you get as a reviewer [6], or is just basically a coin toss even when looking at reviewer groups:

It's a coin toss if paper could get accepted at all, and that's less than ideal but what the system should do (at least) is reject obvious crap, not ensure that something gets clearly accepted. The danger is False Positive (accepted even if it's crap) rather than False Negative (rejected even if it might be something useful).

Overall note: the review system is not ideal and should be improved. But it's a hard, complex and delicate problem.

Re: Government grant-funded research should not be published in for-profit journals

#39
post #4

Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it. It doesn't. Not even close. Peer review doesn't even mean that it's free from errors, free from fraud, free from methodological mischief; it doesn't mean anything at this point. Yet we continue to act like it does. Darwin's work wasn't peer reviewed. Nor Einstein's. It's something we cooked up in the mid 1…

> Darwin's work wasn't peer reviewed. Nor Einstein's Except it was…? This is absurdly ahistorical and the fact that you cross disciplines in trying to make an incorrect argument questions whether you are in science at all. The structure of peer review in Darwin’s time was different, where experts wrote monographs and gave lectures at symposia that then led to letters among their peers. Which is what happens now, if y…

Sorry. I meant "peer reviewed prior to publication" as the phrase is presently used. I thought that was obvious.

What you're calling "peer review" is what I would call "discussed" or "debated" which it certainly was.

I dispute your claim that the new paradigm is superior.

Re: Government grant-funded research should not be published in for-profit journals

#40
post #4

Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it. It doesn't. Not even close. Peer review doesn't even mean that it's free from errors, free from fraud, free from methodological mischief; it doesn't mean anything at this point. Yet we continue to act like it does. Darwin's work wasn't peer reviewed. Nor Einstein's. It's something we cooked up in the mid 1…

Don't know why you are being downvoted, you are largely correct. I've provided plenty of evidence in another post in this thread showing that journal-based peer review is highly farcical. EDIT: I still want review from a community of scientific peers. I just don't want this review to be in the hands of a tiny number of gatekeepers entangled with journals that largely just slow things down.

Because a lot of people are deeply invested in the present system perhaps? As the article pointed out, there's a lot of money involved, and there are a lot of people who've built their lives around flourishing in the existing system, cut-throat as it may be.
Post reply on HN