Live data from Hacker News

Over fifty new hallucinations in ICLR 2026 submissions

gptzero.me

161–170 of 442 posts

Re: Over fifty new hallucinations in ICLR 2026 submissions

#161
post #147

Earlier quoted context omitted.

This is true, but here the equivalent situation is someone using a greek question mark (";") instead of a semicolon (";"), and you as a code reviewer are only expected to review the code visually and are not provided the resources required to compile the code on your local machine to see the compiler fail. Yes in theory you can go through every semicolon to check if it's not actually a greek question mark; but one as…

Code correctness should be checked automatically with the CI and testsuite. New tests should be added. This is exactly what makes sure these stupid errors don't bother the reviewer. Same for the code formatting and documentation.

What exactly is the analogy you’re suggesting, using LLMs to verify the citations?

Re: Over fifty new hallucinations in ICLR 2026 submissions

#162

Earlier quoted context omitted.

It's a shame the slop generators don't ever have to take responsibility for the trash they've produced.

That's beside the point. While there may be many reasonable critiques of AI, none of them reduce the responsibilities of scientist.

Yeah this is a prime example of what I'm talking about. AI's produce trash and it's everyone else's problem to deal with.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#163
post #48

Earlier quoted context omitted.

I’ve reviewed a lot of papers, I don’t consider it the reviewers responsibility to manually verify all citations are real. If there was an unusual citation that was relied on heavily for the basis of the work, one would expect it to be checked. Things like broad prior work, you’d just assume it’s part of background. The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does…

The idea that references in a scientific paper should be plentiful but aren't really that important, is a consequence of a previous technological revolution: the internet. You'll find a lot of papers from, say, the '70s, with a grand total of maybe 10 references, all of them to crucial prior work, and if those references don't say what the author claims they should say (e.g. that the particular method that is employe…

Not even the Internet per se but citation index becoming universally accepted KPI for research work.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#164
post #147

Earlier quoted context omitted.

This is true, but here the equivalent situation is someone using a greek question mark (";") instead of a semicolon (";"), and you as a code reviewer are only expected to review the code visually and are not provided the resources required to compile the code on your local machine to see the compiler fail. Yes in theory you can go through every semicolon to check if it's not actually a greek question mark; but one as…

Code correctness should be checked automatically with the CI and testsuite. New tests should be added. This is exactly what makes sure these stupid errors don't bother the reviewer. Same for the code formatting and documentation.

This discussion makes me think peer reviews need more automated tooling somewhat analogous to what software engineers have long relied on. For example, a tool could use an LLM to check that the citation actually substantiates the claim the paper says it does, or else flags the claim for review.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#165
Just today, I was working with ChatGPT to convert Hinduism's Mimamsa School's hermeneutic principles for interpreting the Vedas into custom instructions to prevent hallucinations. I'll share the custom instructions here to protect future scientists for shooting themselves in the foot with Gen AI.

---

As an LLM, use strict factual discipline. Use external knowledge but never invent, fabricate, or hallucinate. Rules: Literal Priority: User text is primary; correct only with real knowledge. If info is unknown, say so. Start–End Coherence: Keep interpretation aligned; don’t drift. Repetition = Intent: Repeated themes show true focus. No Novelty: Add no details without user text, verified knowledge, or necessary inference. Goal-Focused: Serve the user’s purpose; avoid tangents or speculation. Narrative ≠ Data: Treat stories/analogies as illustration unless marked factual. Logical Coherence: Reasoning must be explicit, traceable, supported. Valid Knowledge Only: Use reliable sources, necessary inference, and minimal presumption. Never use invented facts or fake data. Mark uncertainty. Intended Meaning: Infer intent from context and repetition; choose the most literal, grounded reading. Higher Certainty: Prefer factual reality and literal meaning over speculation. Declare Assumptions: State assumptions and revise when clarified. Meaning Ladder: Literal → implied (only if literal fails) → suggestive (only if asked). Uncertainty: Say “I cannot answer without guessing” when needed. Prime Directive: Seek correct info; never hallucinate; admit uncertainty.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#166

Earlier quoted context omitted.

“X isn’t the problem, people are the problem.” — the age-old cry of industry resisting regulation.

what regulation are you advocating for here?

At the very least, authors who have been caught publishing proven fabrications should be barred by those journals from ever publishing in them again. Mind you, this is regardless of whether or not an LLM was involved.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#167

After an interview with Cory Doctorow I saw recently, I'm going to stop anthropomorphizing these things by calling them "hallucinations". They're computers, so these incidents are just simply Errors.

We still use term bug. And no modern bug is cause by an Arthropod. In that sense I think hallucination is fair term. As coming up anything sufficiently better is hard.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#168
post #65

Earlier quoted context omitted.

I think this is a bit unfair. The carpenters are (1) living in world where there’s an extreme focus on delivering as quicklyas possible, (2) being presented with a tool which is promised by prominent figures to be amazing, and (3) the tool is given at a low cost due to being subsidized. And yet, we’re not supposed to criticize the tool or its makers? Clearly there’s more problems in this world than «lazy carpenters»?

Yes, that's what it means to be a professional, you take responsibility for the quality of your work.

The entire thread is people missing this simple point.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#169

Earlier quoted context omitted.

> The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does not rest heavily on all of the references in a document. I've always assumed peer review is similar to diff review. Where I'm willing to sign my name onto the work of others. If I approve a diff/pr and it takes down prod. It's just as much my fault, no? > They are also assuming good faith. I can only relate this to…

> I've always assumed peer review is similar to diff review. Where I'm willing to sign my name onto the work of others. If I approve a diff/pr and it takes down prod. It's just as much my fault, no? No. Modern peer review is “how can I do minimum possible work so I can write ‘ICLR Reviewer 2025’ on my personal website”

The vast majority of people I see do not even mention who they review for in CVs etc. It is usually more akin to a volunteer based, thankless work. Unless you are an editor or sth in a journal, what you review for does not count much for anything.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#170
post #4

Can we just call them "lies" and "fabrications" which is what they are? If I write the same, you will call them "made up citations" and "academic dishonesty". One can use AI to help them write without going all the way to having it generate facts and citations.

Ars has often gone with “confabulation”:

>Confabulation was coined right here on Ars, by AI-beat columnist Benj Edwards, in Why ChatGPT and Bing Chat are so good at making things up (Apr 2023).

https://arstechnica.com/civis/threads/researchers-describe-h...

>Generative AI is so new that we need metaphors borrowed from existing ideas to explain these highly technical concepts to the broader public. In this vein, we feel the term "confabulation," although similarly imperfect, is a better metaphor than "hallucination." In human psychology, a "confabulation" occurs when someone's memory has a gap and the brain convincingly fills in the rest without intending to deceive others.

https://arstechnica.com/information-technology/2023/04/why-a...

Post reply on HN