Live data from Hacker News

Government grant-funded research should not be published in for-profit journals

experimental-history.com

101–110 of 154 posts

Re: Government grant-funded research should not be published in for-profit journals

#101
post #56

Earlier quoted context omitted.

> IMO, academics that do this are not very competent, because ... Where's the cry-laugh emoji when I need it. Of course academics check where stuff is published. Please... There are still real journals put there, although you might not know which is which.

Ah, look, another smug sneer that ignores the evidence I presented, and makes another circular argument (i.e. that because academics look at rep, this is justified, even though I provided evidence disputing this). I know what journals are better / not. But reputation only is helpful in letting you ignore trash journals, once you are out of trash land, rep is just not a very meaningful factor, and you have to focus on…

Where's the evidence you presented?

What are some higher-profile journals that are in fact less trustworthy in many ways?

Re: Government grant-funded research should not be published in for-profit journals

#102
post #89
post #53

Earlier quoted context omitted.

> This is actually what ruined my respect for Academia. Spoken like someone who never went through grad school at a competitive R1 program It was already a grueling 60-80 hour grind every week with frequent all nighters, high-pressure deadlines, absolute minimal pay, thankless duties, and plenty of politics. It's about the same for professors too. We already paid our dues by helping peer review (for free) a half doze…

I don’t think folks in academia have come to terms with how much the above attitude has completely and nearly entirely undermined the credibility of the entire scientific and academic community in the eyes of the general public. You don’t need a degree to understand how much utter junk science is being published by those who think they are superior to you. Just read a few actual papers end to end and look at the data…

Go read /r/LawyerTalk and enjoy the horror of the dawning realization that this is a lot of professionals. I think it's an issue that stems from getting too deep into the minutiae of the technical and cultural matters of one's field; you become a really good scientist, or lawyer, or SWE (by the standards of scientists and lawyers and SWEs), and end up coming to conclusions that everyone outside the bubble looks at and says, "That's absolutely asinine." Well, laymen just don't understand the details, you know? (Even though the whole point of these professions is to provide services to laymen, fix problems laymen come to them with, and guide laymen to make practical and logical decisions when a $500/hr appointment isn't called for.)

These people take themselves too seriously, and other people only take them seriously when there are material ramifications for not doing so. Otherwise, they're viewed as pompous busy-bodies and don't do themselves any favors by playing to the role.

Re: Government grant-funded research should not be published in for-profit journals

#103

Earlier quoted context omitted.

You've provided evidence that prominent journals experience retractions, fraudulent results, etc. All true. But it is not the job of peer reviewers to decide what gets published. You've provided evidence that peer-reviewed science often turns out to be incomplete, inaccurate, wrong, fraudulent etc. All true. But it is not the job of peer reviewers to assure completeness, accuracy, or freedom from fraud. A peer review…

> A peer reviewer reads a paper and make comments on it. That's it! They don't check primary data, they don't investigate methods, they don't interrogate scientists, they don't re-run experiments just to double check. They assist a journal's editors in editing--that's it. Um, what? I have done all these things in reviews, and know other academics that have done these things as well. More confusingly though, if you ar…

Would be interesting if you would be willing to share a paper you reviewed and detail your review process of it. I don't see how one could check primary data or interrogate scientists in a blind review process, for example.

Re: Government grant-funded research should not be published in for-profit journals

#104

Earlier quoted context omitted.

Ah, look, another smug sneer that ignores the evidence I presented, and makes another circular argument (i.e. that because academics look at rep, this is justified, even though I provided evidence disputing this). I know what journals are better / not. But reputation only is helpful in letting you ignore trash journals, once you are out of trash land, rep is just not a very meaningful factor, and you have to focus on…

Where's the evidence you presented? What are some higher-profile journals that are in fact less trustworthy in many ways?

I literally said it was posted in this thread, and a quick Ctrl+F of my username on this page would have found you it in a half second: https://news.ycombinator.com/item?id=47249236

Re: Government grant-funded research should not be published in for-profit journals

#105

Earlier quoted context omitted.

> A peer reviewer reads a paper and make comments on it. That's it! They don't check primary data, they don't investigate methods, they don't interrogate scientists, they don't re-run experiments just to double check. They assist a journal's editors in editing--that's it. Um, what? I have done all these things in reviews, and know other academics that have done these things as well. More confusingly though, if you ar…

Would be interesting if you would be willing to share a paper you reviewed and detail your review process of it. I don't see how one could check primary data or interrogate scientists in a blind review process, for example.

This is IMO just bad faith sealioning, you can look at the whole replication crisis in psychology and social science (esp. the work of people like Nick Brown and the GRIM test, or Uri Simonsohn), or sites like Retraction Watch, and see clear evidence of everything I am saying. There are endless papers in ML research going into issues with test datasets and data duplication, etc. In plenty of cases all data and code is made open, so it is trivial to check data issues and methods.

Also, review is back and forth, and has rounds: you almost always interrogate the scientists of the paper you are reviewing, this almost like the definition of peer review. I don't think you have any idea of what you are talking about at all.

EDIT: Heck, just hop on over to https://openreview.net/ and take a look at the whole review process for some random paper (e.g. https://openreview.net/forum?id=cp5PvcI6w8_)

Re: Government grant-funded research should not be published in for-profit journals

#106
post #10

We already have open-access publications: Just put it on arXiV. Most researchers I work with do this already. The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything. TPCs don't use it in their list of citations, neither do grant funding agencies or government institutions. The current academic enterprise relies heavily on third-party gatekeeping.…

I've long wished that "journals" and academic societies would transition from a publishing model to a cultivation model. If everything is available on arXiv, that's great, but it also means the best of the best is mixed in with all the rest.

Journals (in the sense of whoever is on the editorial board) don't need to cease to exist; they just need to transition to "here's our list this month of what the best new articles are on X topic". The paper's already there on arXiv, you could already read it before. But having a group of editors that cultivate a list of good articles (as well as the peer review process that can, in an ideal world, serve to improve a paper) can serve to make sifting through arXiv less overwhelming, and draw attention to papers in particular subfields, subject matter, or whatever other criteria might be relevant.

Re: Government grant-funded research should not be published in for-profit journals

#107
post #10

We already have open-access publications: Just put it on arXiV. Most researchers I work with do this already. The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything. TPCs don't use it in their list of citations, neither do grant funding agencies or government institutions. The current academic enterprise relies heavily on third-party gatekeeping.…

Unfortunately I think charging money is a necessary signal that this particular gatekeeper is doing a good job. We should recognise that money is a necessary part of this process, else there is no gate to keep. But we shpuld reverse the economics by having people pay to get their stuff peer reviewed. Imagine if reviewing research papers was something you could get paid to do, the incentive then isnt to rubber stamp t…

> But we shpuld reverse the economics by having people pay to get their stuff peer reviewed.

Not really. There would be perverse incentives where the publisher benefits from accepting more articles. For good journals that would be a conflict of interests at best where they would optimise the marketing-to-acceptance ratio. I can’t believe I am writing something good about scientific publisher, but at least when the reader pays they are incentivise to publish things that have an audience. Otherwise, they are going to cut corners, and I mean more than they currently do. And it’s not hypothetical, there are already terrible publishers doing this.

Re: Government grant-funded research should not be published in for-profit journals

#108
post #22

> So the solution here is straightforward: every government grant should ... People who write such sentences have no idea what they are talking about or are being intentionally naive for whatever reason. Just because your one-sentence solution reads simple doesn't make the actual solution simple. Because such a solution involves changes to laws, changes to entrenched interests, changes to distribution of money involv…

This defeatist attitude is why we can't have nice things anymore. Fun fact, all of those things happened and this is already government policy for any NSF grant: https://www.nsf.gov/policies/document/faq-public-access So maybe consider that when you give up on obvious things that are good based on some conspiracy theory that the "man" is trying to keep you down, what you're actually doing is being part of the system…

Why would you claim that my statement is defeatist? It is just acknowledging the reality of how things work nowadays without saying anything about giving up. If anything, it cautions people that change will not come easily and to be prepared for it. I vote, give money to political/public causes, to go meetings of my federal representatives, post my opinions on HN, ... I am just being more realistic about my expectations.

Re: Government grant-funded research should not be published in for-profit journals

#109
post #10

We already have open-access publications: Just put it on arXiV. Most researchers I work with do this already. The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything. TPCs don't use it in their list of citations, neither do grant funding agencies or government institutions. The current academic enterprise relies heavily on third-party gatekeeping.…

Unfortunately I think charging money is a necessary signal that this particular gatekeeper is doing a good job. We should recognise that money is a necessary part of this process, else there is no gate to keep. But we shpuld reverse the economics by having people pay to get their stuff peer reviewed. Imagine if reviewing research papers was something you could get paid to do, the incentive then isnt to rubber stamp t…

The problem is that this becomes a race to the bottom of actual quality and turns into advertising.

Sponsored reviews of products are basically this. If you are paying a reviewer for a stamp of approval and the reviewer sets the bar too high, why would you want to pay that reviewer? On the other end of the reviewer, it's easy to get more money by providing that stamp of approval to more people--not fewer--so they're incentivized to make it fairly easy to achieve.

Re: Government grant-funded research should not be published in for-profit journals

#110
post #10

We already have open-access publications: Just put it on arXiV. Most researchers I work with do this already. The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything. TPCs don't use it in their list of citations, neither do grant funding agencies or government institutions. The current academic enterprise relies heavily on third-party gatekeeping.…

I've long wished that "journals" and academic societies would transition from a publishing model to a cultivation model. If everything is available on arXiv, that's great, but it also means the best of the best is mixed in with all the rest. Journals (in the sense of whoever is on the editorial board) don't need to cease to exist; they just need to transition to "here's our list this month of what the best new articl…

I don't see any reason why we shouldn't make this transitive. working professionals track the literature. if there were a standard way to publish a "I think this paper is interesting" signal, then we could roll that all up. there are certainly practitioners that I really do trust to be in the game for the right reasons, if they think a paper represents a contribution, then that's a strong signal for me.
Post reply on HN