Wow! They're literally submitting references to papers by Firstname Lastname, John Doe and Jane Smith and nobody is noticing or punishing them.
To be honest, this one could just as well be a sloppy bibliography.
GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
461–470 of 528 posts
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#462I spot-checked one of the flagged papers (from Google, co-authored by a colleague of mine) The paper was https://openreview.net/forum?id=0ZnXGzLcOg and the problem flagged was "Two authors are omitted and one (Kyle Richardson) is added. This paper was published at ICLR 2024." I.e., for one cited paper, the author list was off and the venue was wrong. And this citation was mentioned in the background section of the pa…
>this error does make me pause to wonder how much of the rest of the paper used AI assistance And this is what's operative here. The error spotted, the entire class of error spotted, is easily checked/verified by a non-domain expert. These are the errors we can confirm readily, with obvious and unmistakable signature of hallucination. If these are the only errors, we are not troubled. However: we do not know if these…
> However: we do not know if these are the only errors, they are merely a signature that the paper was submitted without being thoroughly checked for hallucinations
Given how stupidly tedious and error-prone citations are, I have no trouble believing that the citation error could be the only major problem with the paper, and that it's not a sign of low quality by itself. It would be another matter entirely if we were talking about something actually important to the ideas presented in the paper, but it isn't.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#463Earlier quoted context omitted.
For ML/AI/Comp sci articles, providing reproducible code is a great option. Basically, PoC or GTFO.
The most annoying ones are those which discuss loosely the methodology but then fail to publish the weights or any real algorithms. It's like buying a piece of furniture from IKEA, except you just get an Allen key, a hint at what parts to buy, and blurry instructions.
This is so egregious. The value of such papers is basically nothing but they're extremely common.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#464I was getting completely AI-generated reviews for a WACV publication back in 2024. The area chairs are so overworked that authors don't have much recourse, which sucks but is also really hard to handle unless more volunteers step up to the bat to help organize the conference. (If you're qualified to review papers, please email the program chair of your favorite conference and let them know -- they really need the hel…
This is 100% negligence.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#465NeurIPS leadership doesn’t think hallucinated references are necessarily disqualifying; see the full article from Fortune for a statement from them: https://archive.ph/yizHN > When reached for comment, the NeurIPS board shared the following statement: “The usage of LLMs in papers at AI conferences is rapidly evolving, and NeurIPS is actively monitoring developments. In previous years, we piloted policies regarding th…
> the content of the papers themselves are not necessarily invalidated. For example, authors may have given an LLM a partial description of a citation and asked the LLM to produce bibtex (a formatted reference) Maybe I'm overreacting, but this feels like an insanely biased response. They found the one potentially innocuous reason and latched onto that as a way to hand-wave the entire problem away. Science already had…
I found at least one example[0] of authors claiming the reason for the hallucination was exactly this. That said, I do think for this kind of use, authors should go to the effort of verifying the correctness of the output. I also tend to agree with others who have commented that while a hallucinated citation or two may not be particularly egregious, it does raise concerns about what other errors may have been missed.
[0] https://openreview.net/forum?id=IiEtQPGVyV¬eId=W66rrM5XPk
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#466Earlier quoted context omitted.
Im really not motivated by this argument; it seems a false equivalence. Its not merely a spell checker or removing some tedium. As a professional mathematician I used wikipedia all the time to lookup quick facts before verifying it myself or elsewhere. A calculator well; I can use an actual programming language. Up until this point neither of those tools were asvertised or used by people to entirely replace human inp…
I hate to sound like a 19 year old on Reddit but: AI People: "AI is a completely unprecedented technology where its introduction is unlike the introduction of any other transformative technology in history! We must treat it totally differently!" Also AI People: "You're worried about nothing, this is just like when people were worried about the internet."
The internet analogy is apt because it was in fact a massive bubble, but that bubble popping didn't mean the tech went away. Same will happen again, which is a point both extremes miss. One would have you believe there is no bubble and you should dump all your money into this industry, while the other would have us believe that once the bubble pops all this AI stuff will be debunked and discarded as useless scamware.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#467Earlier quoted context omitted.
One issue with this analogy is that calculators really are precise when used correctly. LLMs are not. I do think they can be used in research but not without careful checking. In my own work I’ve found them most useful as search aids and brainstorming sounding boards.
One issue with this analogy is that paper encyclopedias really are precise when used correctly. Wikipedia is not. I do think it can be used in research but not without careful checking. In my own work I've found it most useful as a search aid and for brainstorming. ^ this same comment 10 years ago
Paper encyclopedias were neither precise nor accurate. You could count on them to give you ballpark figures most of the time, but certainly not precise answers. And that's assuming the set was new, but in reality most encyclopedias ever encountered by people in reality were several years old at least. I remember the encyclopedia set I had access to in the 90s was written before the USSR fell..
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#468Earlier quoted context omitted.
Autotranslating technical texts is very hard. After the translation, you muct check that all the technical words were translated correctly, instead of a fancy synonym that does not make sense. (A friend has an old book translated a long time ago (by a human) from Russian to Spanish. Instead of " complex numbers ", the book calls them " complicated numbers ". :) )
I remember one time when I had written a bunch of user facing text for an imaging app and was reviewing our French translation. I don't speak French but I was pretty sure "plane" (as in geometry) shouldn't be translated as "avion". And this was human translated!
You'd be surprised how shoddy human translations can be, and it's not necessarily because of the translators themselves.
Typically what happens is that translators are given an Excel sheet with the original text in a column, and the translated text must be put into the next column. Because there's no context, it's not necessarily clear to the translator whether the translation for plane should be avion (airplane) or plan (geometric plane). The translator might not ever see the actual software with their translated text.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#469Earlier quoted context omitted.
I don't necessarily disagree, but researchers are not required to be good communicators. An academic can lead their field and be a terrible lecturer. A specialist can let a generalist help explain concepts for them. They should still review the final result though. There is no excuse for not doing that.
I disagree here. A good researcher has to be a good communicator. I am not saying that it is necessarily the case that you don't understand the topic if you cannot explain it well enough to someone new, but it is essential to communicate to have a good exchange of ideas with others, and consequently, become a better researcher. This is one of the skills you learn in a PhD program.
That is how it should be, yes. Do PhDs always meet that standard though? No.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#470Earlier quoted context omitted.
I would argue that an LLM is a perfectly sensible tool for structure-preserving machine translation from another language to English. (Where by "another language", you could also also substitute "very poor/non-fluent English." Though IMHO that's a bit silly, even though it's possible; there's little sense in writing in a language you only half know, when you'd get a less-lossy result from just writing in your native…
Autotranslating technical texts is very hard. After the translation, you muct check that all the technical words were translated correctly, instead of a fancy synonym that does not make sense. (A friend has an old book translated a long time ago (by a human) from Russian to Spanish. Instead of " complex numbers ", the book calls them " complicated numbers ". :) )
idk I think Gemini 2.5 did a great job at almost all research math papers translating from french to english...