No mercy to brain slugs.
New arXiv policy: 1-year ban for hallucinated references
161–170 of 242 posts
Re: New arXiv policy: 1-year ban for hallucinated references
#162There needs be to a careful vetting before such adverse actions. If somebody includes a name and pushed it without express permission, does everyone get the ban? I agree that implemented the right way, this is good.
Plus afaik you can add any co-author you want without validation. So you can ban everyone on arxiv with one paper with one sentence.
As I mentioned in another thread:
To be a coauthor on a preprint that you have not submitted, you have to actively "claim" it (using a password given to the author who submitted). It's on you to double-check before claiming.
I surely hope that only "confirmed" coauthors will get the ban, it's only logical.
Re: New arXiv policy: 1-year ban for hallucinated references
#163Earlier quoted context omitted.
No. One single hallucinated citation on a document with you as an author is not evidence of your reckless disregard for anything. These exaggerations are crazy and you would absolutely deny such accusations if you missed your co-author's AI hallucinating a citation on your manuscript too. At best it would be careless , if you really relish extrapolating from one data point and smearing people's character based on tha…
arXiV is not intended to be your blog. You should be held to a zero-mistake standard when publishing academic work. The people I worry for are the junior researchers who are going to be splash damage for dishonest PIs. The PIs, though, deserve everything that’s coming for them.
Maybe I'm misunderstanding you, but zero-mistake seems harsh. I would say that AI references are a sign of something that is not simply a mistake.
However, we can have zero tolerance for certain techniques for "writing" a paper. Plagiarism and inventing data are already examples of this, if there is evidence for these techniques being used there is no excuse. We could say the same for AI references - any writing process that could produce these is by definition not a technique we want.
So the mistake isn't not checking a reference the AI gave. The mistake is letting the AI make references for you.
If we agree that academic research is important then I think we can impose certain standards on how you do it. We can dissalow certain tools if that means we can't trust the output. Just like an electrician can't use certain techniques, even if they're easy, because we don't trust the final result.
Re: New arXiv policy: 1-year ban for hallucinated references
#164Seeing the usual LLM hypers angry replying to this on twitter is such a tell. Just like the comments on the LLM poisoning articles, some people just can't accept that some people don't like LLMs and get upset when you put any amount of hindrance to their rapid acceptance.
It's hard for me to even understand their perspective. Researching references for a published academic paper isn't some incidental busywork task, it's supposed to be a core part of doing research which is the core of the job. If you don't have sympathy for someone who, say, paid a person on Fiverr to cook up a paper rather than writing it themselves and then didn't even bother to check the references, why is using an LLM and not checking any better?
Re: New arXiv policy: 1-year ban for hallucinated references
#165> The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue. This is incredibly good for science. arXiv is free, but it's a privilege not a right! I'm not seeing this clearly listed on https://info.arxiv.org/help/policies/index.html so it's possible this is planned but not live yet - or perhaps I'm not digging deeply…
> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's abili…
It’s easy to avoid this whole issue: write the paper yourself.
Re: New arXiv policy: 1-year ban for hallucinated references
#166Earlier quoted context omitted.
You're saying it as if the poor author just had no choice but to let LLM write their bibliography. To avoid hallucinations, maybe just don't let an LLM write any part of your paper? You can only get in this situation if you let a bullshit generator write your paper, and the fraud is that you are generating bullshit and calling it a paper. No buts. It's impossible to trigger this accidentally, or without reckless disr…
Calling LLMs "bullshit generators" in the year 2026 just shows a lack of seriousness.
And yet people are trying to defend LLM-generated made-up bullshit citations in scientific papers.
Re: New arXiv policy: 1-year ban for hallucinated references
#167how will they detect hallucinated refs at scale? Manual spot checks? Automated DOI verification? The policy seems right but enforcement is the hard part.
Enforcement is secondary and is allowed to take weeks / months / never at all if nobody reads the paper. It's about being able to ban if an issue arrises; not about keeping the database strictly clean.
Re: New arXiv policy: 1-year ban for hallucinated references
#168Earlier quoted context omitted.
I bet, since this has been posted, someone here has already vibe coded a reference checker that they plan to put behind a subscription. This is good for reference checking, but I doubt this will do much for the most likely shoddy science that accompanies hallucinated references.
The frontier LLMs are getting pretty good at checking this sort of thing. You could prompt them to not only verify the references are real but that they actually state what the article claims. Some human review will still be needed but I'll bet this approach could find a lot of academic fraud.
why is the standard response to "this tech isn't reliable enough for this" to run its output through the same unreliable tech?
The device-fixer started breaking devices instead of fixing them. Tell it to fix itself!
Re: New arXiv policy: 1-year ban for hallucinated references
#169Earlier quoted context omitted.
You're saying it as if the poor author just had no choice but to let LLM write their bibliography. To avoid hallucinations, maybe just don't let an LLM write any part of your paper? You can only get in this situation if you let a bullshit generator write your paper, and the fraud is that you are generating bullshit and calling it a paper. No buts. It's impossible to trigger this accidentally, or without reckless disr…
Calling LLMs "bullshit generators" in the year 2026 just shows a lack of seriousness.
Not really - much of work consists of what David Graeber described as “bullshit jobs”. Now AI and its backers are proposing to automate all that bullshit.
Re: New arXiv policy: 1-year ban for hallucinated references
#170Earlier quoted context omitted.
> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's abili…
In science, one hallucinated reference can corrupt the entire rest of the work. So you're completely wrong.
And every piece of work in future which cites the paper with the hallucinated reference.