Is there a comparison to rate of reference errors in other forums?
GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
11–20 of 528 posts
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#12It is very concerning that these hallucinations passed through peer review. It's not like peer review is a fool-proof method or anything, but the fact that reviewers did not check all references and noticed clearly bogus ones is alarming and could be a sign that the article authors weren't the only ones using LLMs in the process...
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#13Could you run a similar analysis for pre-2020 papers? It'd be interesting to know how prevalent making up sources was before LLMs.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#14This suggests that nobody was screening this papers in the first place—so is it actually significant that people are using LLMs in a setting without meaningful oversight?
These clearly aren't being peer-reviewed, so there's no natural check on LLM usage (which is different than what we see in work published in journals).
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#15Which is worse: a) p-hacking and suppressing null results b) hallucinations c) falsifying data Would be cool to see an analysis of this
All 3 of these should be categorized as fraud, and punished criminally.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#16If these are so easy to identify, why not just incorporate some kind of screening into the early stages of peer review?
Because real work takes time and effort, and there is no real incentive for it here.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#17If these are so easy to identify, why not just incorporate some kind of screening into the early stages of peer review?
What makes you believe that are easy to identify?
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#18[deleted]
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#19A lot of research in AI/ML seems to me to be "fake it and never make it". Literally it's all about optics, posturing, connections, publicity. Lots of bullshit and little substance. This was true before AI slop, too.
But the fact that AI slop can make it pass the review really showcases how much a paper's acceptance hinges on things, other than the substance and results of the paper.
I even know PIs who got fame and funding based on some research direction that supposedly is going to be revolutionary. Except all they had were preliminary results that from one angle, if you squint, you can envision some good result. But then the result never comes. That's why I say, "fake it, and never make it".
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#20Wow! They're literally submitting references to papers by Firstname Lastname, John Doe and Jane Smith and nobody is noticing or punishing them.
They might (I hope) still be punished after discovery.