Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New arXiv policy: 1-year ban for hallucinated references

twitter.com

Post reply on HN
Print view View original

181–190 of 242 posts

Re: New arXiv policy: 1-year ban for hallucinated references

#181

Post by redsocksfan45 »

dataflow wrote: post #18
btown wrote: post #4

> The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue. This is incredibly good for science. arXiv is free, but it's a privilege not a right! I'm not seeing this clearly listed on https://info.arxiv.org/help/policies/index.html so it's possible this is planned but not live yet - or perhaps I'm not digging deeply…

> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's abili…

[dead]

Re: New arXiv policy: 1-year ban for hallucinated references

#182

Post by redsocksfan45 »

DiogenesKynikos wrote: post #128
toast0 wrote: post #63

Earlier quoted context omitted.

If your co-author inserted the fradulent reference, I agree that you may not have committed fraud. But your co-author did, and you didn't check their work. and knowing that you didn't check their work, you signed off on it. You didn't pick your co-author very well, but arXiv lacks investigative powers to determine which co-author did the bad, so they all get the consequence.

Do you think every co-author on a 100-author paper checks every citation? It's like saying that every member of a large software team personally reviews every line of code. It's just completely divorced from reality.

[dead]

Re: New arXiv policy: 1-year ban for hallucinated references

#183

Post by wallst07 »

casey2 wrote: post #115
eqvinox wrote: post #112

Earlier quoted context omitted.

You don't need to be actively enforcing a rule 100% on everyone. Speed cameras don't cover every stretch of road either. It's enough for them to place this policy and enforce it when they become aware of violations. Someone reading the slopped paper (or, here, trying to follow a reference) will notice sooner or later. > Being required to publish in a peer reviewed journal will close off arxiv for many researchers for…

Even acts that would be criminal in the US occur less in China due to properly enforced fines. Nobody does things assuming they will get caught unless there is a high likelihood of getting caught. Research and practice has shown that the strongest deterrent is certainty.

Impact = Risk * Probability Occurrence.

If the fine is high enough (risk) , but probability low, people will not do the thing because of the impact on them.

Re: New arXiv policy: 1-year ban for hallucinated references

#184

Post by netdevphoenix »

nradov wrote: post #118
nomel wrote: post #96

Earlier quoted context omitted.

I bet, since this has been posted, someone here has already vibe coded a reference checker that they plan to put behind a subscription. This is good for reference checking, but I doubt this will do much for the most likely shoddy science that accompanies hallucinated references.

The frontier LLMs are getting pretty good at checking this sort of thing. You could prompt them to not only verify the references are real but that they actually state what the article claims. Some human review will still be needed but I'll bet this approach could find a lot of academic fraud.

Your approach is good for catching stuff that human reviewers might miss not as a first line default-only unit. The whole reason this is happening is because humans are not doing their job. Your solution (humans not doing their job) is just increasing the scope of the problem.

Re: New arXiv policy: 1-year ban for hallucinated references

#185

Post by emil-lp »

withinboredom wrote: post #154
godelski wrote: post #100

Earlier quoted context omitted.

> In an ideal world, one would be keeping notes on references used In a far less than ideal world authors are referencing papers they've at least read the title and abstract of. In an ideal world, authors would be only referencing works they have read in their entirety. I don't think we need to live in the ideal world[0], but let's also not pretend the ideal world is even remotely out of reach. Let's also be honest t…

Even as a human, you can still fuck up references. I submitted a paper with a reference author as Elisio because I couldn’t read my own handwriting. After submitting, I double checked all the references through an LLM. It pointed out that their name was actually Enrique. Yes, you should probably double check your references before submitting, not after. Point is, I didn’t even trust the LLM at first. But after verify…

Sounds like you could use a tool like Zotero.

With proper bibliography management tools, everything (that has one) is centered around the DOI.

In fact, if a DOI is present, it's trivial to verify authors, title, venue, year, pages etc.

Of course, some older and more obscure papers won't have a DOI, but the vast majority of research work has.

Re: New arXiv policy: 1-year ban for hallucinated references

#186

Post by emil-lp »

lou1306 wrote: post #159
incognition wrote: post #151

Earlier quoted context omitted.

coauthors about to get roasted

To be a coauthor on a preprint that you have not submitted, you have to actively "claim" it (using a password given to the author who submitted). It's on you to double-check before claiming.

Is that your definition or theirs?

I can't see that in the code of conduct.

Re: New arXiv policy: 1-year ban for hallucinated references

#187

Post by boccaff »

tdeck wrote: post #164
noobermin wrote: post #60

Seeing the usual LLM hypers angry replying to this on twitter is such a tell. Just like the comments on the LLM poisoning articles, some people just can't accept that some people don't like LLMs and get upset when you put any amount of hindrance to their rapid acceptance.

It's hard for me to even understand their perspective. Researching references for a published academic paper isn't some incidental busywork task, it's supposed to be a core part of doing research which is the core of the job. If you don't have sympathy for someone who, say, paid a person on Fiverr to cook up a paper rather than writing it themselves and then didn't even bother to check the references, why is using an…

There is a lot of "throw it against the wall, and if it sticks, write it up" empirical work against benchmarks. It leads to post-hoc rationalization of the work and browser plugins using LLMs to find references for work that is already written. It is a bureaucratic view about "you need a citation for this", where people misunderstand the citation as a checkbox, instead of "you need to substantiate this claim, as I, the reviewer, do not accept this as a fact".

Re: New arXiv policy: 1-year ban for hallucinated references

#188

Post by eqvinox »

dataflow wrote: post #135
eqvinox wrote: post #116

Earlier quoted context omitted.

I do believe this policy is appropriate to deal with the reckless disregard of posting hallucinated references. It's a conscious decision to not take the time to check your AI output, and instead waste a whole bunch of other people's time letting them essentially do that for you in duplicate. Feels like that should disqualify you from participation for a bit. Intent or no intent.

> Feels like that should disqualify you from participation for a bit. Intent or no intent. Exactly! For a bit ! Yet this is not for a bit! This is a lifetime disqualification , and that's been my entire grip the whole time! Is nobody reading this? "The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue ."

Mhm. Okay, honestly, I maybe don't have enough data to judge how much impact that requirement has.

I also haven't seen anything on how this works with multiple authors, which could go anywhere from draconian to weakening the entire thing.

Re: New arXiv policy: 1-year ban for hallucinated references

#189

Post by [deleted] »

gizajob wrote: post #169
DiogenesKynikos wrote: post #129

Earlier quoted context omitted.

Calling LLMs "bullshit generators" in the year 2026 just shows a lack of seriousness.

Not really - much of work consists of what David Graeber described as “bullshit jobs”. Now AI and its backers are proposing to automate all that bullshit.

[deleted]

Re: New arXiv policy: 1-year ban for hallucinated references

#190

Post by mpalmer »

small_scombrus wrote: post #138
nomel wrote: post #127

Earlier quoted context omitted.

> The frontier LLMs are getting pretty good at checking this sort of thing. No, this is career ending high stakes. it requires old school "actually check a record of reality" type methods, like a database query or http get to one of the many services that hold this info.

I think they're saying that frontier LLMs may be usable to spot citations that are correct by shape (a real citation) but incorrect by usage (unrelated to the text) I kind of hate the idea, but you probably could do a lazy LLM check of every paper and every citation and have it flag possible wrong (second sense) citations for human review But you'd need a LOT of tokens and a LOT of human-hours

> have it flag possible wrong (second sense) citations for human review

And then what, we're done? How have we avoided the need for the same exhaustive human review? It only saves human review time if you trust the LLM not to miss things.

Post reply on HN