Hurray!
New arXiv policy: 1-year ban for hallucinated references
141–150 of 242 posts
Re: New arXiv policy: 1-year ban for hallucinated references
#142Earlier quoted context omitted.
I do believe this policy is appropriate to deal with the reckless disregard of posting hallucinated references. It's a conscious decision to not take the time to check your AI output, and instead waste a whole bunch of other people's time letting them essentially do that for you in duplicate. Feels like that should disqualify you from participation for a bit. Intent or no intent.
> Feels like that should disqualify you from participation for a bit. Intent or no intent. Exactly! For a bit ! Yet this is not for a bit! This is a lifetime disqualification , and that's been my entire grip the whole time! Is nobody reading this? "The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue ."
That doesn't sound like a lifetime ban to me.
Re: New arXiv policy: 1-year ban for hallucinated references
#143Earlier quoted context omitted.
A "mistake" would be a typo in a real citation. A hallucinated citation is evidence of just plain laziness and negligence, which taints the entire submission.
No it is not. Seriously. All you need for this to happen is for your lab partner to ask AI to add a missing citation that they are already familiar with at the last minute before a midnight submission deadline, and for the AI to hallucinate something else, and for them to honestly miss this. It does not even imply any involvement on your part, let alone that either of you were lazy or negligent on the actual research…
This is like saying lawyers should be allowed to submit AI-hallucinated case references or quotations in court documents. Because by your logic, that, too, should be perfectly acceptable. Yet is not, for hopefully obvious reasons. Why exactly should scientific research be any different? If your paper contains hallucinated references, we can't verify your assertions in the paper, and therefore must question the paper as a whole.
Re: New arXiv policy: 1-year ban for hallucinated references
#144Seeing the usual LLM hypers angry replying to this on twitter is such a tell. Just like the comments on the LLM poisoning articles, some people just can't accept that some people don't like LLMs and get upset when you put any amount of hindrance to their rapid acceptance.
It's also hilarious that they complain about this because, from what I've seen, most LLM hypers will talk about something being irrelevant or taken over by AI with no understanding of what that something really is or involves.
Re: New arXiv policy: 1-year ban for hallucinated references
#145> The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue. This is incredibly good for science. arXiv is free, but it's a privilege not a right! I'm not seeing this clearly listed on https://info.arxiv.org/help/policies/index.html so it's possible this is planned but not live yet - or perhaps I'm not digging deeply…
> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's abili…
In science, one hallucinated reference can corrupt the entire rest of the work. So you're completely wrong.
Re: New arXiv policy: 1-year ban for hallucinated references
#146Earlier quoted context omitted.
How are you suggesting the fake citation came about? Why are you writing papers and not having actually read the source you took the material from?
> Why are you writing papers and not having actually read the source you took the material from? They're explicitly not writing papers. The fake citations are created and inserted by the LLM
They are still purposely writing a paper, whether that is with the help of an LLM or not. They are instructing the LLM to do the task of finding citations. It's no difference to googling for a paper that explains a specific point. You would still double check Google's output.
Re: New arXiv policy: 1-year ban for hallucinated references
#147Earlier quoted context omitted.
> Feels like that should disqualify you from participation for a bit. Intent or no intent. Exactly! For a bit ! Yet this is not for a bit! This is a lifetime disqualification , and that's been my entire grip the whole time! Is nobody reading this? "The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue ."
That doesn't sound like a lifetime ban to me.
What? Then how long are they disqualifying them from submitting prior to acceptance, if not lifetime? It certainly doesn't say 1 year or something.
Re: New arXiv policy: 1-year ban for hallucinated references
#148Earlier quoted context omitted.
The frontier LLMs are getting pretty good at checking this sort of thing. You could prompt them to not only verify the references are real but that they actually state what the article claims. Some human review will still be needed but I'll bet this approach could find a lot of academic fraud.
> The frontier LLMs are getting pretty good at checking this sort of thing. No, this is career ending high stakes. it requires old school "actually check a record of reality" type methods, like a database query or http get to one of the many services that hold this info.
LLMs can make tool calls to do database and http queries to search for, buy, and cross reference a citation.
Re: New arXiv policy: 1-year ban for hallucinated references
#149Earlier quoted context omitted.
This puts the burden to make sure it's right on the submitter, where it should be. Verification can come at any time after that; the submitter understands the consequences of hallucinated references. Verification can be crowd-sourced (and likely will be). Nothing stops someone from putting a PDF on the internet. I'm fine with ArXiv holding a high standard.
More than fine, let’s encourage it. We deserve it, it’s one of the ways to differentiate from the Elsevier et al shitboxes!
Not to mention Zenodo, Academia.edu, etc.
Re: New arXiv policy: 1-year ban for hallucinated references
#150Earlier quoted context omitted.
That doesn't sound like a lifetime ban to me.
What? Then how long are they disqualifying them from submitting prior to acceptance, if not lifetime? It certainly doesn't say 1 year or something.
> Then how long are they disqualifying them from submitting prior to acceptance, if not lifetime?
Well, "lifetime ban" means "you are not allowed back in". Their ban specifically allows you back in (after a specified period) subject to fulfilling a single constraint.
It's conditional acceptance back in, which is not the same as a lifetime ban which is unconditional.