Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Over fifty new hallucinations in ICLR 2026 submissions

gptzero.me

Post reply on HN
Print view View original

181–190 of 442 posts

Re: Over fifty new hallucinations in ICLR 2026 submissions

#181

Post by qbit42 »

MarkusQ wrote: post #68

This is as much a failing of "peer review" as anything. Importantly, it is an intrinsic failure, which won't go away even if LLMs were to go away completely. Peer review doesn't catch errors. Acting as if it does, and thus assuming the fact of publication (and where it was published) are indicators of veracity is simply unfounded. We need to go back to the food fight system where everyone publishes whatever they want…

I don’t think many researchers take peer review alone as a strong signal, unless it is a venue known for having serious reviewing (e.g. in CS theory, STOC and FOCS have a very high bar). But it acts as a basic filter that gets rid of obvious nonsense, which on its own is valuable. No doubt there are huge issues, but I know my papers would be worse off without reviewer feedback

Re: Over fifty new hallucinations in ICLR 2026 submissions

#182

Post by bjourne »

grayhatter wrote: post #71
andy99 wrote: post #48

Earlier quoted context omitted.

I’ve reviewed a lot of papers, I don’t consider it the reviewers responsibility to manually verify all citations are real. If there was an unusual citation that was relied on heavily for the basis of the work, one would expect it to be checked. Things like broad prior work, you’d just assume it’s part of background. The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does…

> The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does not rest heavily on all of the references in a document. I've always assumed peer review is similar to diff review. Where I'm willing to sign my name onto the work of others. If I approve a diff/pr and it takes down prod. It's just as much my fault, no? > They are also assuming good faith. I can only relate this to…

For ICLR reviewers were asked to review 5 papers in two weeks. Unpaid voluntary work in addition to their normal teaching, supervision, meetings, and other research duties. It's just not possible to understand and thoroughly review each paper even for topic experts. If you want to compare peer review to coding, it's more like "no syntax errors, code still compiles" rather than pr review.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#183

Post by SauntSolaire »

thfuran wrote: post #178
SauntSolaire wrote: post #153

Earlier quoted context omitted.

That's beside the point. While there may be many reasonable critiques of AI, none of them reduce the responsibilities of scientist.

>While there many reasonable critiques of AI But you just said we weren’t supposed to criticize the purveyors of AI or the tools themselves.

No, I merely said that the scientist is the one responsible for the quality of their own work. Any critiques you may have for the tools which they use don't lessen this responsibility.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#184

Post by simonw »

I'm finding the GPTZero share links difficult to understand. Apparently this one shows a hallucinated citation but I couldn't understand what it was trying to tell me: https://app.gptzero.me/documents/9afb1d51-c5c8-48f2-9b75-250...

(I'm on mobile, haven't looked on desktop.)

Re: Over fifty new hallucinations in ICLR 2026 submissions

#185

Post by freehorse »

grayhatter wrote: post #71
andy99 wrote: post #48

Earlier quoted context omitted.

I’ve reviewed a lot of papers, I don’t consider it the reviewers responsibility to manually verify all citations are real. If there was an unusual citation that was relied on heavily for the basis of the work, one would expect it to be checked. Things like broad prior work, you’d just assume it’s part of background. The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does…

> The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does not rest heavily on all of the references in a document. I've always assumed peer review is similar to diff review. Where I'm willing to sign my name onto the work of others. If I approve a diff/pr and it takes down prod. It's just as much my fault, no? > They are also assuming good faith. I can only relate this to…

A reviewer is assessing the relevance and "impact" of a paper rather than correctness itself directly. Reviewers may not even have access to the data itself that authors may have used. The way it essentially works is an editor asks the reviewers "is this paper worthy to be published in my journal?" and the reviewers basically have to answer that question. The process is actually the editor/journal's responsibility.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#186

Post by neilv »

proto-n wrote: post #53
neilv wrote: post #49

https://blog.iclr.cc/2025/11/19/iclr-2026-response-to-llm-ge... > Papers that make extensive usage of LLMs and do not disclose this usage will be desk rejected. This sounds like they're endorsing the game of how much can we get away with, towards the goal of slipping it past the reviewers , and the only penalty is that the bad paper isn't accepted. How about "Papers suspected of fabrications, plagiarism, ghost writer…

1. "Suspected" is just that, suspected, you can't penalize papers based on your gut feel 2. LLM-s are a tool, and there's nothing wrong with using them unless you misuse them

"Suspected" doesn't necessarily mean only gut feel.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#187

Post by zzzeek »

andy99 wrote: post #48
CapitalistCartr wrote: post #40

Earlier quoted context omitted.

I'm an industrial electrician. A lot of poor electrical work is visible only to a fellow electrician, and sometimes only another industrial electrician. Bad technical work requires technical inspectors to criticize. Sometimes highly skilled ones.

I’ve reviewed a lot of papers, I don’t consider it the reviewers responsibility to manually verify all citations are real. If there was an unusual citation that was relied on heavily for the basis of the work, one would expect it to be checked. Things like broad prior work, you’d just assume it’s part of background. The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does…

correct me if I'm wrong but citations in papers follow a specific format, and the case here is that a tool was used to validate that they are all real. Certainly a tool that scans a paper for all citations and verifies that they actually exist in the journals they reference shouldn't be all that technically difficult to achieve?

Re: Over fifty new hallucinations in ICLR 2026 submissions

#188

Post by thfuran »

SauntSolaire wrote: post #183
thfuran wrote: post #178

Earlier quoted context omitted.

>While there many reasonable critiques of AI But you just said we weren’t supposed to criticize the purveyors of AI or the tools themselves.

No, I merely said that the scientist is the one responsible for the quality of their own work. Any critiques you may have for the tools which they use don't lessen this responsibility.

>No, I merely said that the scientist is the one responsible for the quality of their own work.

No, you expressed unqualified agreement with a comment containing

“And yet, we’re not supposed to criticize the tool or its makers?”

>Any critiques you may have for the tools which they use don't lessen this responsibility.

People don’t exist or act in a vacuum. That a scientist is responsible for the quality of their work doesn’t mean that a spectrometer manufacture that advertises specs that their machines can’t match and induces universities through discounts and/or dubious advertising claims to push their labs to replace their existing spectrometers with new ones which have many bizarre and unexpected behaviors including but not limited to sometimes just fabricating spurious readings has made no contribution to the problem of bad results.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#189

Post by zzzeek »

tpoacher wrote: post #87
pbhjpbhj wrote: post #62

Earlier quoted context omitted.

Surely there are tools to retrieve all the citations, publishers should spot it easily. However the paper is submitted, like a folder on a cloud drive, just have them include a folder with PDFs/abstracts of all the citations? They might then fraudulently produce papers to cite, but they can't cite something that doesn't exist.

how delightfully optimistic of you to think those abstracts would not also be ai generated ...

sure but then the citations are no longer "hallucinated", they actually point to something fraudulent. that's a different problem.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#190

Post by bigstrat2003 »

theoldgreybeard wrote: post #16

If a carpenter builds a crappy shelf “because” his power tools are not calibrated correctly - that’s a crappy carpenter, not a crappy tool. If a scientist uses an LLM to write a paper with fabricated citations - that’s a crappy scientist. AI is not the problem, laziness and negligence is. There needs to be serious social consequences to this kind of thing, otherwise we are tacitly endorsing it.

> If a carpenter builds a crappy shelf “because” his power tools are not calibrated correctly - that’s a crappy carpenter, not a crappy tool.

It's both. The tool is crappy, and the carpenter is crappy for blindly trusting it.

> AI is not the problem, laziness and negligence is.

Similarly, both are a problem here. LLMs are a bad tool, and we should hold people responsible when they blindly trust this bad tool and get bad results.

Post reply on HN