Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New arXiv policy: 1-year ban for hallucinated references

twitter.com

Post reply on HN
Print view View original

171–180 of 242 posts

Re: New arXiv policy: 1-year ban for hallucinated references

#171

Post by gucci-on-fleek »

mks_shuffle wrote: post #94

While this is certainly a welcome step, I hope there is more work done to fix the underlying problem of easily creating correct BibTeX entries for the cited papers. Citations for any given paper can come from a wide range of journals with various publishers, conferences, and preprints. The same paper can be available from multiple sources with varying details, e.g. arXiv and the conference website. Tools like Zotero…

Fun fact: if an article has a DOI, you can just use curl to get a BibTeX entry. An example using one of my articles:

  $ curl -L "https://doi.org/10.47397/tb/43-1/tb133chernoff-widows" -H 'Accept: application/x-bibtex'
  @article{Chernoff_2022, title={Automatically removing widows and orphans with lua-widow-control}, volume={43}, ISSN={0896-3207}, url={http://dx.doi.org/10.47397/tb/43-1/tb133chernoff-widows}, DOI={10.47397/tb/43-1/tb133chernoff-widows}, number={1}, journal={TUGboat}, publisher={TeX Users Group}, author={Chernoff, Max}, year={2022}, pages={28–39} }
This is the exact same method that Zotero uses internally, so this won't ever give you better results, but I still find it kinda neat.

Re: New arXiv policy: 1-year ban for hallucinated references

#172

Post by Foivos »

I just wish to anyone who is against this policy to be forced to review a paper that turns out to be unedited AI slop. Reviewers are experts volunteers who do it for free. It is incredibly frustrating to have spent 4 hours reading a paper where you try your best to make sense of what the authors are trying to prove just to realize that it is hallucinations.

The authors should value the time of the reviewers higher than their own time. So, if you include AI nonsense in your paper, it is insulting.

Re: New arXiv policy: 1-year ban for hallucinated references

#173

Post by gizajob »

cyclecycle wrote: post #153

This has become such a problem in scholarly publishing that we have a business that provides citation checking https://groundedai.company/ that we've been buidling for a couple of years now

What’s the hallucination rate of your AI?

Re: New arXiv policy: 1-year ban for hallucinated references

#175

Post by gizajob »

kingstnap wrote: post #14
pinkmuffinere wrote: post #11

Earlier quoted context omitted.

> think proof of Riemann Hypothesis autonomously generated (end to end) formally proven - is it allowed or not? Sorry to be rude, but this seems like a dumb question. I want science to progress. A primary purpose of these journals is to progress science. A full proof of the Riemann Hypothesis progresses science. I don't care how it was produced, if Hitler is coauthor, etc, I just care that it is correct. Whether the…

Terence Tao had a nice talk from the Future of Mathematics conference posted yesterday [0] that shapes a lot of my own feelings on this matter. The short of it is he argues how first to correctness shouldn't be the only goal / isn't a great optimisation incentive. Presentation and digestibility of correct results is a missing 1/3 when you've finished generation and verification. I completely agree with him. You don't…

He would say that now he’s got tenure though.

Re: New arXiv policy: 1-year ban for hallucinated references

#176

Post by brazzy »

dumpsterdiver wrote: post #90
dataflow wrote: post #57

Earlier quoted context omitted.

No. One single hallucinated citation on a document with you as an author is not evidence of your reckless disregard for anything. These exaggerations are crazy and you would absolutely deny such accusations if you missed your co-author's AI hallucinating a citation on your manuscript too. At best it would be careless , if you really relish extrapolating from one data point and smearing people's character based on tha…

I’ve disagreed with some of your other stances in this thread, but I want to acknowledge the validity of your take here. You’re right that a single hallucinated line is not evidence of reckless disregard - because that could have happened on a final follow-up pass after you had performed due diligence. It’s happened to me. I know how challenging it can be to keep bad patterns out of LLM generated output, because huma…

> You’re right that a single hallucinated line is not evidence of reckless disregard

It absolutely is.

> - because that could have happened on a final follow-up pass after you had performed due diligence.

A "final follow-up pass" that lets the LLM make whatever changes it deems appropriate completely negates all the due diligence you did before, unless you very carefully review the diffs. And a new or substantially changed citation should stand out in that diff so much that there's no possible excuse to missing it.

> It’s happened to me.

Then you were guilty of reckless disregard.

> I know how challenging it can be to keep bad patterns out of LLM generated output

If your research paper contains any LLM generated output you did not manually vet, you are a hack and should not get published.

Re: New arXiv policy: 1-year ban for hallucinated references

#177

Post by soraminazuki »

It's not unexpected, but still sad to see so many comments opposing even the smallest step against low-effort fraud in academic publications. Is this what hacker culture has been reduced to in the age of the slop era? Open hostility against science and engineering?

Re: New arXiv policy: 1-year ban for hallucinated references

#178

Post by brazzy »

dataflow wrote: post #24
ajkjk wrote: post #21

Earlier quoted context omitted.

It's not the kind of mistake that is possible unless you're engaging in fraud anyway.

> It's not the kind of mistake that is possible unless you're engaging in fraud anyway. Seriously? You can't fathom an honest researcher asking for AI to find a citation they know exists, and the AI inserting or modifying a citation incorrectly without them realizing? If you find evidence of fraud by all means lay down the hammer. Using a single hallucinated citation like it's some kind of ironclad proxy just because…

> Seriously? You can't fathom an honest researcher asking for AI to find a citation they know exists, and the AI inserting or modifying a citation incorrectly without them realizing?

Indeed I cannot. If you do that, you are not, in fact, an honest researcher. You're a lazy hack.

Re: New arXiv policy: 1-year ban for hallucinated references

#180

Post by gammalost »

djoldman wrote: post #88
rgmerk wrote: post #67

Good. If it’s not worth your time to check the output of your LLM carefully, it’s not worth my time to read it.

Unfortunately, it's probably not worth your time to read 99% of arxiv papers, LLM generated or otherwise. Ever pick a random one and really dive in?

Well, yeah, 99% of arXiv papers were not written for me or you. They were written for someone who works in a niche within a niche. That's (in my view) the beauty of research.
Post reply on HN