Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New arXiv policy: 1-year ban for hallucinated references

twitter.com

Post reply on HN
Print view View original

91–100 of 242 posts

Re: New arXiv policy: 1-year ban for hallucinated references

#91

Post by gpm »

ksd482 wrote: post #79
toast0 wrote: post #33

Earlier quoted context omitted.

> It's just one darn hallucinated citation for heaven's sake, not fraud or something. It is fraud. > It doesn't account for the substance or quality of their work at all. References are part of the work. If you're making up the references, what else are you making up? > People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's ability to progress…

> It is fraud. I think we are talking semantics here. While fraud does require intention to deceive, I get the sentiment that hallucinated citations shouldn't be dismissed as simply carelessness. It should be something stronger than that: gross negligence or something MUCH stronger! There should absolutely be repercussions for this. But let's not call it fraud. That word is reserved for something specific. EDIT: some…

The intent to deceive is there. The deception is lying when you submit it that it is a scholarly piece of work in which amongst many other things you know the citations are accurate. This false representation was knowingly and intentionally made at the time of submission.

The citation being incorrect is merely the proof of deception not the (relevant) deception itself.

Fraud is the correct description provided (and this is practically a guarantee) you intended to benefit from the submission of the paper (e.g. by bolstering your resume).

Re: New arXiv policy: 1-year ban for hallucinated references

#92

Post by lbrito »

noobermin wrote: post #60

Seeing the usual LLM hypers angry replying to this on twitter is such a tell. Just like the comments on the LLM poisoning articles, some people just can't accept that some people don't like LLMs and get upset when you put any amount of hindrance to their rapid acceptance.

Crazy that this is graytexted. So basically HN consensus is that we need to be hyper and accelerate llm adoption everywhere.

Bonkers. At the same time peak hn

Re: New arXiv policy: 1-year ban for hallucinated references

#93

Post by cozzyd »

AnimalMuppet wrote: post #87
asdff wrote: post #36

Earlier quoted context omitted.

There are no deadlines for journal submissions. Even if you felt you were running close to your revisions being due, an email to an editor will probably fix this for you. And what you described is still negligent, not verifying the garbage output bot did not in fact output garbage.

Even more, there are no deadlines for arXiv submissions.

Kinda. There is a deadline to appear in the next days posting and this can be important if e.g. you want to get something on the arxiv before a talk or a proposal deadline.

Re: New arXiv policy: 1-year ban for hallucinated references

#94

Post by mks_shuffle »

While this is certainly a welcome step, I hope there is more work done to fix the underlying problem of easily creating correct BibTeX entries for the cited papers. Citations for any given paper can come from a wide range of journals with various publishers, conferences, and preprints. The same paper can be available from multiple sources with varying details, e.g. arXiv and the conference website. Tools like Zotero have certainly made it significantly easier to extract citations from webpages of publication, but I still find issues with the extracted BibTeX details. While author names and titles are often extracted correctly, I still have to manually ensure that details like publication venue, year, volume number, page number, URL, etc. are extracted correctly and also shown correctly in LaTeX format. Different publications can use different citation styles. This can unfortunately lead to taking shortcuts with AI-generated citation data due to the lack of an easy and unified approach to extract consistent citation data. I am not sure whether hallucinated citations are being generated in the main manuscript or in a separate BibTeX file, so I may be a bit off in my understanding.

Re: New arXiv policy: 1-year ban for hallucinated references

#95

Post by cozzyd »

miki123211 wrote: post #35
squirrelon wrote: post #15

Had a colleague submit a paper with literal AI slop left in the text, got hit with a nasty revision request. Check your drafts before you submit, people. The reviewers will find it.

Also check your LaTeX comments, Arxiv makes those publicly visible!!! I'm a screen reader user and usually read papers as raw TeX. I've seen everything: slurs, demeaning comments towards reviewers and professors, admissions of fraud, instructions to coauthors to commit further fraud before paper submission to mask the earlier fraud... it's all there. There's far less of it than I would think, definitely I think it wo…

That's why my forarxiv make target includes a run through latexpand

Re: New arXiv policy: 1-year ban for hallucinated references

#96

Post by nomel »

btown wrote: post #4

> The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue. This is incredibly good for science. arXiv is free, but it's a privilege not a right! I'm not seeing this clearly listed on https://info.arxiv.org/help/policies/index.html so it's possible this is planned but not live yet - or perhaps I'm not digging deeply…

I bet, since this has been posted, someone here has already vibe coded a reference checker that they plan to put behind a subscription.

This is good for reference checking, but I doubt this will do much for the most likely shoddy science that accompanies hallucinated references.

Re: New arXiv policy: 1-year ban for hallucinated references

#97

Post by vasco »

MinimalAction wrote: post #30

There needs be to a careful vetting before such adverse actions. If somebody includes a name and pushed it without express permission, does everyone get the ban? I agree that implemented the right way, this is good.

Plus afaik you can add any co-author you want without validation. So you can ban everyone on arxiv with one paper with one sentence.

Re: New arXiv policy: 1-year ban for hallucinated references

#98

Post by tardedmeme »

dumpsterdiver wrote: post #90
dataflow wrote: post #57

Earlier quoted context omitted.

No. One single hallucinated citation on a document with you as an author is not evidence of your reckless disregard for anything. These exaggerations are crazy and you would absolutely deny such accusations if you missed your co-author's AI hallucinating a citation on your manuscript too. At best it would be careless , if you really relish extrapolating from one data point and smearing people's character based on tha…

I’ve disagreed with some of your other stances in this thread, but I want to acknowledge the validity of your take here. You’re right that a single hallucinated line is not evidence of reckless disregard - because that could have happened on a final follow-up pass after you had performed due diligence. It’s happened to me. I know how challenging it can be to keep bad patterns out of LLM generated output, because huma…

You're saying it as if the poor author just had no choice but to let LLM write their bibliography. To avoid hallucinations, maybe just don't let an LLM write any part of your paper?

You can only get in this situation if you let a bullshit generator write your paper, and the fraud is that you are generating bullshit and calling it a paper. No buts. It's impossible to trigger this accidentally, or without reckless disregard for the truth.

Re: New arXiv policy: 1-year ban for hallucinated references

#99

Post by rgmerk »

djoldman wrote: post #88
rgmerk wrote: post #67

Good. If it’s not worth your time to check the output of your LLM carefully, it’s not worth my time to read it.

Unfortunately, it's probably not worth your time to read 99% of arxiv papers, LLM generated or otherwise. Ever pick a random one and really dive in?

Agreed. There was already too much human generated slop in academia.

And I’m not talking about good faith research that didn’t pan out, I mean research that is completely useless for any other purpose other than convincing a casual observer that the authors are doing research.

Re: New arXiv policy: 1-year ban for hallucinated references

#100

Post by godelski »

toast0 wrote: post #33
dataflow wrote: post #18

Earlier quoted context omitted.

> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's abili…

> It's just one darn hallucinated citation for heaven's sake, not fraud or something. It is fraud. > It doesn't account for the substance or quality of their work at all. References are part of the work. If you're making up the references, what else are you making up? > People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's ability to progress…

  > In an ideal world, one would be keeping notes on references used
In a far less than ideal world authors are referencing papers they've at least read the title and abstract of. In an ideal world, authors would be only referencing works they have read in their entirety. I don't think we need to live in the ideal world[0], but let's also not pretend the ideal world is even remotely out of reach. Let's also be honest that in the current setting a lot of citations are being used to encourage a work be accepted more than they are being used because of their utility to the paper. The average ML paper now is 8 pages and has >50 citations. That's crazy

[0] References can be entire textbooks, which is potentially too high of a bar

Post reply on HN