The problem is the people doing this. The AI doesn't have agency, it's just an autocomplete tool.
Right so what solutions do you have here to prevent people from using the autocomplete tool for harm?
AI intensifies fight against ‘paper mills’ that churn out fake research
11–20 of 186 posts
Re: AI intensifies fight against ‘paper mills’ that churn out fake research
#12The problem is the people doing this. The AI doesn't have agency, it's just an autocomplete tool.
Guns don’t kill people. They have no agency.
But they sure make it easier.
Re: AI intensifies fight against ‘paper mills’ that churn out fake research
#13The upside from this downside is that it increases the performance of papers that can be replicated easily. In fact, once AI can replicate papers, that'll be helpful for verifying a lot of research.
How will AI be able to replicate papers?
This seems like a bleak dystopian fiction concept actually. AIs being in control of human experimentation on the basis of well meaning but poorly interpreted ethical guidelines (with, of course, fully hallucinated loopholes).
Maybe add in that this has become one of the few sources of income for people as more and more jobs have been lost to the machines so many are compelled to participate.
Re: AI intensifies fight against ‘paper mills’ that churn out fake research
#14Instead of fighting against "paper mills", let’s fight against journals. There are strong arguments against the need for peer-review for example [1]. Science does not get worse when there are more bad papers, science gets better when there are more good papers. Most papers in AI aren’t even reviewed by peers or editor and guess what: we have lots of progress happening. I’m not saying that is because the lack of review, I’m saying that reviews are not necessary for progress. Furthermore, was the iPhone good because it was reviewed by a board of "independent" reviewers? No it wasn’t. Let’s just ignore Nature. Good papers are good because they have good arguments and if they are not good, then time will tell. Papers are not good simply because Nature (TM) put a stamp on it.
[1]: https://www.experimental-history.com/p/science-is-a-strong-l...
Re: AI intensifies fight against ‘paper mills’ that churn out fake research
#15You get what you incentivize. We incentivize papers so we get papers. Also anything used as a metric tends to cease to be a good metric.
I think the main problem is not in itself incentivizing papers or using papers as a metric. The problem is that they're in many ways the only thing incentivized and the only metric given serious consideration. And the general expectations for/form of a paper are so homogenized across a given field. This makes it very feasible to game the system in a particular way that is bad for science.
It's also a serious misallocation of incentives purely from a system functioning perspective. At the very least you need to incentivize peer review alongside incentivizing papers. Currently there is hardly any incentive at all to help carefully review papers, so we have no checks and balances in place to deal with the natural result of incentivizing the writing of papers. If there were greater heterogeneity in scientific roles it wouldn't be so bad for some to have heavy paper incentives -- because there'd be others with heavy incentives to do a good job highlighting the best papers and finding flaws in the bad ones.
Re: AI intensifies fight against ‘paper mills’ that churn out fake research
#16Feels like AI makes the fight against paper mills easier. A group could submit AI-generated papers to different places and create a public index based on how successful they are at getting their nonsense papers accepted.
Re: AI intensifies fight against ‘paper mills’ that churn out fake research
#17You get what you incentivize. We incentivize papers so we get papers. Also anything used as a metric tends to cease to be a good metric.
There is a good metric in AI research: reproducible results on well known benchmarks (Big Bench for example).
Re: AI intensifies fight against ‘paper mills’ that churn out fake research
#18Not even a mention of the galactica model. I suspect the author probably isn't aware of its existence..
I didn’t know what the Galactica Model was.
https://www.technologyreview.com/2022/11/18/1063487/meta-lar...
Re: AI intensifies fight against ‘paper mills’ that churn out fake research
#19The problem is the people doing this. The AI doesn't have agency, it's just an autocomplete tool.
Right so what solutions do you have here to prevent people from using the autocomplete tool for harm?
Legitimate journals (or groups of journals) might need to institute a throttling mechanism to limit the rate at which individual authors can submit papers. They might also need some kind of reputation tracking system so that reviewers can flag articles as low quality, and then future submissions by the same author will be treated as lower priority. Open access journals can raise author fees.
Re: AI intensifies fight against ‘paper mills’ that churn out fake research
#20Feels like AI makes the fight against paper mills easier. A group could submit AI-generated papers to different places and create a public index based on how successful they are at getting their nonsense papers accepted.
Your comment seems self-contradictory.
AI makes the fight against spam harder since it is harder to detect fakes. The group would only get more successful over time. The “AI detection” tools get worse over time and overhyped already anyway, as we learned from the guys running that sci fi submission site.