Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New arXiv policy: 1-year ban for hallucinated references

twitter.com

Post reply on HN
Print view View original

31–40 of 242 posts

Re: New arXiv policy: 1-year ban for hallucinated references

#31

Post by dataflow »

wrs wrote: post #22
dataflow wrote: post #18

Earlier quoted context omitted.

> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's abili…

A "mistake" would be a typo in a real citation. A hallucinated citation is evidence of just plain laziness and negligence, which taints the entire submission.

No it is not. Seriously. All you need for this to happen is for your lab partner to ask AI to add a missing citation that they are already familiar with at the last minute before a midnight submission deadline, and for the AI to hallucinate something else, and for them to honestly miss this. It does not even imply any involvement on your part, let alone that either of you were lazy or negligent on the actual research or substance of the paper. The lack of any sympathy or imagination here is astounding.

Re: New arXiv policy: 1-year ban for hallucinated references

#32

Post by conartist6 »

dataflow wrote: post #18
btown wrote: post #4

> The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue. This is incredibly good for science. arXiv is free, but it's a privilege not a right! I'm not seeing this clearly listed on https://info.arxiv.org/help/policies/index.html so it's possible this is planned but not live yet - or perhaps I'm not digging deeply…

> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's abili…

Yes, it is fraud

Re: New arXiv policy: 1-year ban for hallucinated references

#33

Post by toast0 »

dataflow wrote: post #18
btown wrote: post #4

> The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue. This is incredibly good for science. arXiv is free, but it's a privilege not a right! I'm not seeing this clearly listed on https://info.arxiv.org/help/policies/index.html so it's possible this is planned but not live yet - or perhaps I'm not digging deeply…

> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's abili…

> It's just one darn hallucinated citation for heaven's sake, not fraud or something.

It is fraud.

> It doesn't account for the substance or quality of their work at all.

References are part of the work. If you're making up the references, what else are you making up?

> People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's ability to progress their life or contribute to humanity just because an AI hallucinated a reference one time in their life.

A one year ban is not permanent. Having a negative consequence for making poor decisions seems like an inducement to learn from the mistake?

In an ideal world, one would be keeping notes on references used while doing the research that lead to writing the paper. Choosing not to do that is one poor decision.

Having a positive outlook, if asking an AI to provide references that may have been missed, one should at least verify the references exist and are relevant. Choosing not to do that is also a poor decision, even if one did take notes on references used while researching.

Re: New arXiv policy: 1-year ban for hallucinated references

#34

Post by blazespin »

toast0 wrote: post #33
dataflow wrote: post #18

Earlier quoted context omitted.

> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's abili…

> It's just one darn hallucinated citation for heaven's sake, not fraud or something. It is fraud. > It doesn't account for the substance or quality of their work at all. References are part of the work. If you're making up the references, what else are you making up? > People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's ability to progress…

it's very silly, but not a big deal. Arxiv is becoming irrelevant these days anyways.

In fact would be better if they just banned AI, so we could just get off the luddite platforms.

Automated research is the future, end of story. And really it couldn't have come out at a better time, given the increasingly diminishing returns on human powered research.

Re: New arXiv policy: 1-year ban for hallucinated references

#35

Post by miki123211 »

squirrelon wrote: post #15

Had a colleague submit a paper with literal AI slop left in the text, got hit with a nasty revision request. Check your drafts before you submit, people. The reviewers will find it.

Also check your LaTeX comments, Arxiv makes those publicly visible!!!

I'm a screen reader user and usually read papers as raw TeX. I've seen everything: slurs, demeaning comments towards reviewers and professors, admissions of fraud, instructions to coauthors to commit further fraud before paper submission to mask the earlier fraud... it's all there. There's far less of it than I would think, definitely I think it would be useful to run an LLM anti-fraud pass on the TeX source of all new arxiv papers. It wouldn't catch everything, but it would catch some of the dumbest fraudsters.

On the positive side, you can also find stronger claims that didn't survive review, additional explanations that didn't make the cut due to the conference's page limit, as well as experimental results that the authors felt weren't really worth including. Those need to be approached with an abundance of caution, but are genuinely useful sometimes.

Re: New arXiv policy: 1-year ban for hallucinated references

#36

Post by asdff »

dataflow wrote: post #31
wrs wrote: post #22

Earlier quoted context omitted.

A "mistake" would be a typo in a real citation. A hallucinated citation is evidence of just plain laziness and negligence, which taints the entire submission.

No it is not. Seriously. All you need for this to happen is for your lab partner to ask AI to add a missing citation that they are already familiar with at the last minute before a midnight submission deadline, and for the AI to hallucinate something else, and for them to honestly miss this. It does not even imply any involvement on your part, let alone that either of you were lazy or negligent on the actual research…

There are no deadlines for journal submissions. Even if you felt you were running close to your revisions being due, an email to an editor will probably fix this for you. And what you described is still negligent, not verifying the garbage output bot did not in fact output garbage.

Re: New arXiv policy: 1-year ban for hallucinated references

#37

Post by mianos »

dataflow wrote: post #18
btown wrote: post #4

> The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue. This is incredibly good for science. arXiv is free, but it's a privilege not a right! I'm not seeing this clearly listed on https://info.arxiv.org/help/policies/index.html so it's possible this is planned but not live yet - or perhaps I'm not digging deeply…

> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's abili…

You are being ironic right?

Re: New arXiv policy: 1-year ban for hallucinated references

#38

Post by asdff »

dataflow wrote: post #24
ajkjk wrote: post #21

Earlier quoted context omitted.

It's not the kind of mistake that is possible unless you're engaging in fraud anyway.

> It's not the kind of mistake that is possible unless you're engaging in fraud anyway. Seriously? You can't fathom an honest researcher asking for AI to find a citation they know exists, and the AI inserting or modifying a citation incorrectly without them realizing? If you find evidence of fraud by all means lay down the hammer. Using a single hallucinated citation like it's some kind of ironclad proxy just because…

Why would you ask the ai to find a citation you know exists? Just reach for that citation.

Re: New arXiv policy: 1-year ban for hallucinated references

#39

Post by asdff »

tengwar2 wrote: post #6
bigfishrunning wrote: post #3

Good; academic literature is in crisis because of all of the slop. Forcing some consequences on easily-detectable hallucinations can only be a good thing

It's not just AI, though. I did a doctorate in physics about 40 years back, and bad references were a problem back then.

Imagine how bad they are now then.

Re: New arXiv policy: 1-year ban for hallucinated references

#40

Post by some_furry »

dataflow wrote: post #29
patcon wrote: post #27

Earlier quoted context omitted.

A citation is where you derived knowledge... If you haven't checked it and you are submitting something that should represent a ton of labour (and which will consume labour to review), you don't understand what you're doing. It is not just crossing T's and dotting I'd. Your being set behind is less important than the fact that your publishing is setting everyone else behind. Such a banned person is being helped to "s…

> Your being set behind is less important than the fact that your publishing is setting everyone else behind One hallucinated citation does not in any way imply anyone is being left behind. All it means nobody is checked that particular line of the manuscript after it was written. The rest of the paper could still be solid and treated accordingly. If you find evidence of the contrary, of course treat it accordingly,…

No. It's fraud.
Post reply on HN