Every single person who did this should be censured by their own institutions.
Do it more than once? Lose job.
End of story.
Over fifty new hallucinations in ICLR 2026 submissions
201–210 of 442 posts
Re: Over fifty new hallucinations in ICLR 2026 submissions
#202Earlier quoted context omitted.
> And yet, we’re not supposed to criticize the tool or its makers? Exactly, they're not forcing anyone to use these things, but sometimes others (their managers/bosses) forced them to. Yet it's their responsibility for choosing the right tool for the right problem, like any other professional. If a carpenter shows up to put a roof yet their hammer or nail-gun can't actually put in nails, who'd you blame; the tool, th…
> If a carpenter shows up to put a roof yet their hammer or nail-gun can't actually put in nails, who'd you blame; the tool, the toolmaker or the carpenter? I would be unhappy with the carpenter, yes. But if the toolmaker was constantly over-promising (lying?), lobbying with governments, pushing their tools into the hands of carpenters, never taking responsibility, then I would also criticize the toolmaker. It’s also…
OpenAI and Anthropic at least are both pretty clear about the fact that you need to check the output:
https://openai.com/policies/row-terms-of-use/
https://www.anthropic.com/legal/aup
OpenAI:
> When you use our Services you understand and agree:
Output may not always be accurate. You should not rely on Output from our Services as a sole source of truth or factual information, or as a substitute for professional advice. You must evaluate Output for accuracy and appropriateness for your use case, including using human review as appropriate, before using or sharing Output from the Services. You must not use any Output relating to a person for any purpose that could have a legal or material impact on that person, such as making credit, educational, employment, housing, insurance, legal, medical, or other important decisions about them. Our Services may provide incomplete, incorrect, or offensive Output that does not represent OpenAI’s views. If Output references any third party products or services, it doesn’t mean the third party endorses or is affiliated with OpenAI.
Anthropic:
> When using our products or services to provide advice, recommendations, or in subjective decision-making directly affecting individuals or consumers, a qualified professional in that field must review the content or decision prior to dissemination or finalization. You or your organization are responsible for the accuracy and appropriateness of that information.
So I don't think we can say they are lying.
A poor workman blames his tools. So please take responsibility for what you deliver. And if the result is bad, you can learn from it. That doesn't have to mean not use AI but it definitely means that you need to fact check more thoroughly.
Re: Over fifty new hallucinations in ICLR 2026 submissions
#203"Given that we've only scanned 300 out of 20,000 submissions"
Fuck! 20,000!!
Re: Over fifty new hallucinations in ICLR 2026 submissions
#204Just today, I was working with ChatGPT to convert Hinduism's Mimamsa School's hermeneutic principles for interpreting the Vedas into custom instructions to prevent hallucinations. I'll share the custom instructions here to protect future scientists for shooting themselves in the foot with Gen AI. --- As an LLM, use strict factual discipline. Use external knowledge but never invent, fabricate, or hallucinate. Rules: L…
Are you sure this even works? My understanding is that hallucinations are a result of physics and the algorithms at play. The LLM always needs to guess what the next word will be. There is never a point where there is a word that is 100% likely to occur next. The LLM doesn't know what "reliable" sources are, or "real knowledge". Everything it has is user text, there is nothing it knows that isn't user text. It doesn'…
Telling the LLM not to hallucinate reminds me of, "why don't they build the whole plane out of the black box???"
Most people are just lazy and eager to take shortcuts, and this time it's blessed or even mandated by their employer. The world is about to get very stupid.
Re: Over fifty new hallucinations in ICLR 2026 submissions
#205Earlier quoted context omitted.
That is not, cannot be, and shouldn't be, the bar for peer review. There are two major differences between it and code review: 1. A patch is self-contained and applies to a codebase you have just as much access to as the author. A paper, on the other hand, is just the tip of the iceberg of research work, especially if there is some experiment or data collection involved. The reviewer does not have access to, say, vid…
> That is not, cannot be, and shouldn't be, the bar for peer review. Given the repeatability crisis I keep reading about, maybe something should change? > 2. The software is also self-contained. That's "prodcution". But a scientific paper does not necessarily aim to represent scientific consensus, but a finding by a particular team of researchers. If a paper's conclusions are wrong, it's expected that it will be refu…
IMHO what should change is we stop putting "peer reviewed" articles on a pedestal.
Even if peer review is as rigorous as code reviewed (the former which is usually unpaid), we all know that reviewed code still has bugs, and a programmer would be nuts to go around saying "this code is reviewed by experts, we can assume it's bug free, right?"
But there are too many people who are just assuming peer reviewed articles means they're somehow automatically correct.
Re: Over fifty new hallucinations in ICLR 2026 submissions
#206If a carpenter builds a crappy shelf “because” his power tools are not calibrated correctly - that’s a crappy carpenter, not a crappy tool. If a scientist uses an LLM to write a paper with fabricated citations - that’s a crappy scientist. AI is not the problem, laziness and negligence is. There needs to be serious social consequences to this kind of thing, otherwise we are tacitly endorsing it.
maybe the hammer factory should be held responsible for pumping out so many poorly calibrated hammer
Re: Over fifty new hallucinations in ICLR 2026 submissions
#207If a carpenter builds a crappy shelf “because” his power tools are not calibrated correctly - that’s a crappy carpenter, not a crappy tool. If a scientist uses an LLM to write a paper with fabricated citations - that’s a crappy scientist. AI is not the problem, laziness and negligence is. There needs to be serious social consequences to this kind of thing, otherwise we are tacitly endorsing it.
Absolutely correct. The real issue is that these people can avoid punishment. If you do not care enough about your paper to even verify the existence of citations, then you obviously should not have a job as a scientist.
Taking an academic who does something like that seriously, seem impossible. At best he is someone who is neglecting his most basic duties as an academic, at worst he is just a fraudster. In both cases he should be shunned and excluded.
Re: Over fifty new hallucinations in ICLR 2026 submissions
#208Earlier quoted context omitted.
I'm an industrial electrician. A lot of poor electrical work is visible only to a fellow electrician, and sometimes only another industrial electrician. Bad technical work requires technical inspectors to criticize. Sometimes highly skilled ones.
I’d love to hear some examples of poor electrical work that you’ve come across that’s often missed or not seen.
I am not an electrician, but when I did projects, I did a lot of research before deciding to hire someone and then I was extremely confused when everyone was proposing doing it slightly differently.
A lot of them proposed ways that seem to violate the code, like running flex tubing beyond the allowed length or amount of turns.
Another example would be people not accounting for needing fireproof covers if they’re installing recessed, lighting in between dwelling in certain cities…
Heck, most people don’t actually even get the permit. They just do the unpermitted work.
Re: Over fifty new hallucinations in ICLR 2026 submissions
#209This is as much a failing of "peer review" as anything. Importantly, it is an intrinsic failure, which won't go away even if LLMs were to go away completely. Peer review doesn't catch errors. Acting as if it does, and thus assuming the fact of publication (and where it was published) are indicators of veracity is simply unfounded. We need to go back to the food fight system where everyone publishes whatever they want…
No, it's not "as much".
The dominant "failing" here is that this is fraudulent on a professional, intellectual, and moral level.
Re: Over fifty new hallucinations in ICLR 2026 submissions
#210Does anyone know, from a technical standpoint, why are citations such a problem for LLMs? I realize things are probably (much) more complicated than I realize, but programmatically, unlike arbitrary text, citations are generally strings with a well-defined format. There are literally "specs" for citation formats in various academic, legal, and scientific fields. So, naively, one way to mitigate these hallucinations w…
In general, a citation is something that needs to be precise, while LLMs are very good at generating some generic high probability text not grounded in reality. Sure, you could implement a custom fix for the very specific problem of citations, but you cannot solve all kinds of hallucinations. After all, if you could develop a manual solution you wouldn't use an LLM.
There are some mitigations that are used such as RAG or tool usage (e.g. a browser), but they don't completely fix the underlying issue.