Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New arXiv policy: 1-year ban for hallucinated references

twitter.com

Post reply on HN
Print view View original

101–110 of 242 posts

Re: New arXiv policy: 1-year ban for hallucinated references

#101

Post by tardedmeme »

dataflow wrote: post #31
wrs wrote: post #22

Earlier quoted context omitted.

A "mistake" would be a typo in a real citation. A hallucinated citation is evidence of just plain laziness and negligence, which taints the entire submission.

No it is not. Seriously. All you need for this to happen is for your lab partner to ask AI to add a missing citation that they are already familiar with at the last minute before a midnight submission deadline, and for the AI to hallucinate something else, and for them to honestly miss this. It does not even imply any involvement on your part, let alone that either of you were lazy or negligent on the actual research…

In other words: all it needs for your paper to have fraud is for your lab partner to add fraud to your paper.

I'm not seeing the problem here. The only problem is that your lab partner should be banned and not you. But being incentivised to check your co-author's work before submission isn't a bad thing.

Re: New arXiv policy: 1-year ban for hallucinated references

#102

Post by adw »

dataflow wrote: post #57
LPisGood wrote: post #53

Earlier quoted context omitted.

Fraud requires intent to deceive _or_ reckless disregard, sometimes called, “conscious indifference” for the veracity of the statement asserted.

No. One single hallucinated citation on a document with you as an author is not evidence of your reckless disregard for anything. These exaggerations are crazy and you would absolutely deny such accusations if you missed your co-author's AI hallucinating a citation on your manuscript too. At best it would be careless , if you really relish extrapolating from one data point and smearing people's character based on tha…

arXiV is not intended to be your blog. You should be held to a zero-mistake standard when publishing academic work.

The people I worry for are the junior researchers who are going to be splash damage for dishonest PIs. The PIs, though, deserve everything that’s coming for them.

Re: New arXiv policy: 1-year ban for hallucinated references

#103

Post by fc417fc802 »

gpm wrote: post #91
ksd482 wrote: post #79

Earlier quoted context omitted.

> It is fraud. I think we are talking semantics here. While fraud does require intention to deceive, I get the sentiment that hallucinated citations shouldn't be dismissed as simply carelessness. It should be something stronger than that: gross negligence or something MUCH stronger! There should absolutely be repercussions for this. But let's not call it fraud. That word is reserved for something specific. EDIT: some…

The intent to deceive is there. The deception is lying when you submit it that it is a scholarly piece of work in which amongst many other things you know the citations are accurate. This false representation was knowingly and intentionally made at the time of submission. The citation being incorrect is merely the proof of deception not the (relevant) deception itself. Fraud is the correct description provided (and t…

If I violate the letter of the ToS when clicking submit you can correctly argue that I have technically committed fraud! Yet that is almost never what anyone actually means when having discussions like this one.

Fraud in a scientific context generally refers to fabricated research results. At least personally I agree with GP that hallucinated citations are generally something akin to laziness thus not fraud but rather some sort of professional negligence.

Re: New arXiv policy: 1-year ban for hallucinated references

#104

Post by etchalon »

dataflow wrote: post #31
wrs wrote: post #22

Earlier quoted context omitted.

A "mistake" would be a typo in a real citation. A hallucinated citation is evidence of just plain laziness and negligence, which taints the entire submission.

No it is not. Seriously. All you need for this to happen is for your lab partner to ask AI to add a missing citation that they are already familiar with at the last minute before a midnight submission deadline, and for the AI to hallucinate something else, and for them to honestly miss this. It does not even imply any involvement on your part, let alone that either of you were lazy or negligent on the actual research…

You seem incredibly upset you can't get away with fraud and that people are calling it fraud.

Re: New arXiv policy: 1-year ban for hallucinated references

#105

Post by flexagoon »

mixtureoftakes wrote: post #28
dataflow wrote: post #24

Earlier quoted context omitted.

> It's not the kind of mistake that is possible unless you're engaging in fraud anyway. Seriously? You can't fathom an honest researcher asking for AI to find a citation they know exists, and the AI inserting or modifying a citation incorrectly without them realizing? If you find evidence of fraud by all means lay down the hammer. Using a single hallucinated citation like it's some kind of ironclad proxy just because…

if you're not checking citations in the paper youre publishing AND trusting a non SOTA, hallucination prone ai model to come up with sources for it, its probably for the best of everyone that this paper isn't published. yes there will be rare exceptions but in general i feel like this is a really good addition.

> non SOTA, hallucination prone ai model

What SOTA models are not hallucination prone?

Re: New arXiv policy: 1-year ban for hallucinated references

#106

Post by fc417fc802 »

ksd482 wrote: post #79
toast0 wrote: post #33

Earlier quoted context omitted.

> It's just one darn hallucinated citation for heaven's sake, not fraud or something. It is fraud. > It doesn't account for the substance or quality of their work at all. References are part of the work. If you're making up the references, what else are you making up? > People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's ability to progress…

> It is fraud. I think we are talking semantics here. While fraud does require intention to deceive, I get the sentiment that hallucinated citations shouldn't be dismissed as simply carelessness. It should be something stronger than that: gross negligence or something MUCH stronger! There should absolutely be repercussions for this. But let's not call it fraud. That word is reserved for something specific. EDIT: some…

I think (though might well be misunderstanding) that reckless disregard is taken to be an intentional choice but that it does not imply that the outcome itself was intentional. The difference between intentionally doing something that you know for a fact has a high risk of failure but you can't necessarily predict the outcome versus intentionally seeking a particular legally disallowed outcome.

But what LPisGood was saying is that reckless disregard (as opposed to explicit intent) is sufficient to meet the legal bar for fraud.

Re: New arXiv policy: 1-year ban for hallucinated references

#108

Post by flexagoon »

mks_shuffle wrote: post #94

While this is certainly a welcome step, I hope there is more work done to fix the underlying problem of easily creating correct BibTeX entries for the cited papers. Citations for any given paper can come from a wide range of journals with various publishers, conferences, and preprints. The same paper can be available from multiple sources with varying details, e.g. arXiv and the conference website. Tools like Zotero…

Note that Zotero also has a free online tool to generate citations in any format or BibTeX files from a URL/DOI/ISBN/...

https://zbib.org/

Re: New arXiv policy: 1-year ban for hallucinated references

#109

Post by gpm »

fc417fc802 wrote: post #103
gpm wrote: post #91

Earlier quoted context omitted.

The intent to deceive is there. The deception is lying when you submit it that it is a scholarly piece of work in which amongst many other things you know the citations are accurate. This false representation was knowingly and intentionally made at the time of submission. The citation being incorrect is merely the proof of deception not the (relevant) deception itself. Fraud is the correct description provided (and t…

If I violate the letter of the ToS when clicking submit you can correctly argue that I have technically committed fraud! Yet that is almost never what anyone actually means when having discussions like this one. Fraud in a scientific context generally refers to fabricated research results. At least personally I agree with GP that hallucinated citations are generally something akin to laziness thus not fraud but rathe…

Fraud in the scientific world has generally taken the form of fabricated results, but I don't agree that the word has transitioned away from the common and legal meaning of deception in order to get a benefit.

Even if it had though, I'd be perfectly comfortable calling this fraud in this discussion based on the common meaning of the word. Just because we're talking about a scientific context does not mean we need to use the scientific-jargon versions of words - we're not in a scientific context ourselves.

---

And I'd disagree that this is just about the "letter of the ToS". While that is perhaps a necessary component in order to prove the deception, this is really about the cultural expectations of the community that merely happened to have been encoded in the ToS. The fraud would still occur without the ToS, it would merely be next to impossible to show you didn't simply misunderstand the cultural norms and what your actions would lead others to believe.

Post reply on HN