Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Over fifty new hallucinations in ICLR 2026 submissions

gptzero.me

Post reply on HN
Print view View original

401–410 of 442 posts

Re: Over fifty new hallucinations in ICLR 2026 submissions

#401

Post by mike_hearn »

potato3732842 wrote: post #345
ulrashida wrote: post #102

Unfortunately while catching false citations is useful, in my experience that's not usually the problem affecting paper quality. Far more prevalent are authors who mis-cite materials, either drawing support from citations that don't actually say those things or strip the nuance away by using cherry picked quotes simply because that is what Google Scholar suggested as a top result. The time it takes to find these erro…

>These bad actors should be subject to a three strikes rule: the steady corrosion of knowledge is not an accident by these individuals. These people are working in labs funded by Exxon or Meta or Pfizer or whoever and they know what results will make continued funding worthwhile in the eyes of their donors. If the lab doesn't produce the donor will fund another one that will.

No, not really. I've read lots of research papers from commercial firms and academic labs. Bad citations are something I only ever saw in academic papers.

I think that's because a lot of bad citations come from reviewer demands to add more of them during the journal publishing process, so they're not critical to the argument and end up being low effort citations that get copy/pasted between papers. Or someone is just spamming citations to make a weak claim look strong. And all this happens because academic uses citations as a kind of currency (it's a planned non-market economy, so they have to allocate funds using proxy signals).

Commercial labs are less likely to care about the journal process to begin with, and are much less likely to publish weak claims because publishing is just a recruiting tool, not the actual end goal of the R&D department.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#402

Post by mike_hearn »

knome wrote: post #348
bossyTeacher wrote: post #220

Earlier quoted context omitted.

>LLMs can actually make up for their negative contributions. They could go through all the references of all papers and verify them, They will just hallucinate their existence. I have tried this before

I assumed they meant using the LLM to extract the citations and then use external tooling to lookup and grab the original paper, at least verifying that it exists, has relevant title, summary and that the authors are correctly cited.

Which is what the people in this new article are doing.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#403

Post by titanomachy »

Dylan16807 wrote: post #396
nyc_data_geek1 wrote: post #389

Earlier quoted context omitted.

Don't understand why you're being downvoted, here.

Because the second sentence is inflammatory. The side comment is right, it's about low versus high trust societies. Even if GP made a mistake on which names are relevant, they're not being racist about it.

Yes, on looking more closely it’s possible that they made an honest mistake.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#404

Post by mike_hearn »

bryanrasmussen wrote: post #329
hippo22 wrote: post #267

Earlier quoted context omitted.

It seems like this is the type of thing that LLMs would actually excel at though: find a list of citations and claims in this paper, do the cited works support the claims?

sure, except when they hallucinate that the cited works support the claims when they do not. At which point you're back at needing to read the cited works to see if they support the claims.

Sometimes this kind of problem can be fixed by adjusting the prompt.

You don't say "here's a paper, find me invalid citations". You put less pressure on the model by chunking the text into sentences or paragraphs, extracting the citations for that chunk, and presenting both with a prompt like:

The following claim may be evidenced by the text of the article that follows. Please invoke the found_claim tool with a list of the specific sentence(s) in the text that support the claim, or an empty list indicating you could not find support for it in the text.

In other words you make it a needle-in-a-haystack problem, which models are much better at.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#405

Post by mike_hearn »

zzzeek wrote: post #187
andy99 wrote: post #48

Earlier quoted context omitted.

I’ve reviewed a lot of papers, I don’t consider it the reviewers responsibility to manually verify all citations are real. If there was an unusual citation that was relied on heavily for the basis of the work, one would expect it to be checked. Things like broad prior work, you’d just assume it’s part of background. The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does…

correct me if I'm wrong but citations in papers follow a specific format, and the case here is that a tool was used to validate that they are all real. Certainly a tool that scans a paper for all citations and verifies that they actually exist in the journals they reference shouldn't be all that technically difficult to achieve?

It's not, there's lots of ways to resolve citations without even using AI.

I experimented a couple of years ago with getting LLMs to check citations but stopped working on it because there's no incentive. You could run a fancy expensive pipeline burning scarce GPU hours and find a bunch of bad citations. Then what? Nobody cares. No journal is going to retract any of these papers, the academics themselves won't care or even respond to your emails, nobody is willing to pay for this stuff, least of all the universities, journals or governments themselves.

For example, there's a guy in France who runs a pre-LLM pipeline to discover bad papers using hand-coded heuristics like regexs or metadata analysis e.g. checking if a citation has been retracted. Many of the things it detects are plagiarism, paper mills (i.e. companies that sell fake papers to academics for a profit), or the result of joke paper creators like SciGen.

https://dbrech.irit.fr/pls/apex/f?p=9999:1::::::

Other than populating an obscure database nobody knows about, this work achieved bupkis.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#406

Post by [deleted] »

sigmoid10 wrote: post #74
miniwark wrote: post #20

Earlier quoted context omitted.

They explain in the article what they consider a proper citation, an erroneous one and an hallucination, in the section "Defining Hallucitations". They also say than they have many false positives, mostly real papers who are not available online. Thad said, i am also very curious of the result than their tool, would give to papers from the 2010's and before.

If you look at their examples in the "Defining Hallucitations" section, I'd say those could be 100% human errors. Shortening authors' names, leaving out authors, misattributing authors, misspelling or misremembering the paper title (or having an old preprint-title, as titles do change) are all things that I would fully expect to happen to anyone in any field were things get ever got published. Modern tools have made…

[deleted]

Re: Over fifty new hallucinations in ICLR 2026 submissions

#407

Post by mike_hearn »

sigmoid10 wrote: post #74
miniwark wrote: post #20

Earlier quoted context omitted.

They explain in the article what they consider a proper citation, an erroneous one and an hallucination, in the section "Defining Hallucitations". They also say than they have many false positives, mostly real papers who are not available online. Thad said, i am also very curious of the result than their tool, would give to papers from the 2010's and before.

If you look at their examples in the "Defining Hallucitations" section, I'd say those could be 100% human errors. Shortening authors' names, leaving out authors, misattributing authors, misspelling or misremembering the paper title (or having an old preprint-title, as titles do change) are all things that I would fully expect to happen to anyone in any field were things get ever got published. Modern tools have made…

There are other issues. In January they claimed that a US health report contained "fabricated" and "AI generated" citations with the headline being a claim from a Cigna Group report. Their claim it's fabricated is based on nothing more than the URL now being a redirect of the type common in corporate website reorgs.

I did some checking and found the report does exist, but the citation is still not quite correct. Then I discovered someone is running some LLM based citation checker already, which already fact checked this claim and did a correct writeup that seems a lot better than what this GPTZero tool does.

https://checkplease.neocities.org/maha/html/17-loneliness-73...

The mistakes in the citation are the sort of mistake that could have been made by both a human or an AI, really. The visualization in the report is confusing and does contain the 73% number (rounded up), but it's unclear how to interpret the numbers because it's some sort of "vitality index" and not what you'd expect based on how it's introduced. At first glance I actually mis-interpreted it the same way the report does, so it's hard to view this is as clear evidence of AI misuse. Yet the GPTZero folks do make very strong claims based on nothing more than a URL scraper script.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#408

Post by make3 »

WWWWH wrote: post #292

Surely this is gross professional misconduct? If one of my postdocs did this they would be at risk of being fired. I would certainly never trust them again. If I let it get through, I should be at risk. As a reviewer, if I see the authors lie in this way why should I trust anything else in the paper? The only ethical move is to reject immediately. I acknowledge mistakes and so on are common but this is different leag…

Isn't this mostly a set of citation typos? To me this mostly calls for better bibtex checking, writing and checking bibtex is super annoying

Re: Over fifty new hallucinations in ICLR 2026 submissions

#409

Post by make3 »

Isamu wrote: post #22

Someone commented here that hallucination is what LLMs do, it’s the designed mode of selecting statistically relevant model data that was built on the training set and then mashing it up for an output. The outcome is something that statistically resembles a real citation. Creating a real citation is totally doable by a machine though, it is just selecting relevant text, looking up the title, authors, pages etc and pu…

This interpretation would have been ok for old generation models without search tools enabled and without reliable tool use and reasoning. Modern LLMs can actually look up the existence of papers with web search, and with reasoning, one can definitely get reasonable results by requiring the model to double check that everything actually exists.

Re: Over fifty new hallucinations in ICLR 2026 submissions

#410

Post by figassis »

vkou wrote: post #311
dpkirchner wrote: post #265

Earlier quoted context omitted.

Agreed, and I'd go further. If nobody is reviewing citations they may as well not exist. Why bother?

1. To make it clear what is your work, and what is building on someone else's. 2. If the paper turns out to be important, people will bother. 3. There's checking for cursory correctness, and there's forensic torture.

building on imaginary someone else? That's exactly the same as lying. Is a review not about verifying that the paper and even data is correct? I get reviewers can make mistakes, but this seems like defending intentional mistakes.

I mean, in college I have had to review papers, and so took peer review lectures, and nowhere in there was it ever stated that citations are not the reviewer's job. In fact, citation verification was one to the most important parts of the lectures, as in, how to find original sources (when authoring), and how to verify them (when reviewing).

When did peer review get redefined?

Post reply on HN