Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

gptzero.me

Post reply on HN
Print view View original

401–410 of 528 posts

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#401

Post by aydyn »

neilv wrote: post #339
i_am_proteus wrote: post #268

Earlier quoted context omitted.

>this error does make me pause to wonder how much of the rest of the paper used AI assistance And this is what's operative here. The error spotted, the entire class of error spotted, is easily checked/verified by a non-domain expert. These are the errors we can confirm readily, with obvious and unmistakable signature of hallucination. If these are the only errors, we are not troubled. However: we do not know if these…

> If these are the only errors, we are not troubled. However: we do not know if these are the only errors, they are merely a signature that the paper was submitted without being thoroughly checked for hallucinations. They are a signature that some LLM was used to generate parts of the paper and the responsible authors used this LLM without care. I am troubled by people using an LLM at all to write academic research p…

>also plagiarism

To me, this is a reminder of how much of a specific minority this forum is.

Nobody I know in real life, personally or at work, has expressed this belief.

I have literally only ever encountered this anti-AI extremism (extremism in the non-pejorative sense) in places like reddit and here.

Clearly, the authors in NeurIPS don't agree that using an LLM to help write is "plagiarism", and I would trust their opinions far more than some random redditor.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#402

Post by storystarling »

arjvik wrote: post #335
nazgul17 wrote: post #299

Earlier quoted context omitted.

The thing is, when you copy paste a bibliography entry from the publisher or from Google Scholar, the authors won't be wrong. In this case, it is. If I were to write a paper with AI, I would at least manage the bibliography by hand, conscious of hallucinations. The fact that the hallucination is in the bibliography is a pretty strong indicator that the paper was written entirely with AI.

I'm not sure I agree... while I don't ever see myself writing papers with AI, I hate wrangling a bibtex bibliography. I wouldn't trust today's GPT-5-with-web-search to do turn a bullet point list of papers into proper citations without checking myself, but maybe I will trust GPT-X-plus-agent to do this.

This seems solvable today if you treat it as an architecture problem rather than relying on the model's weights. I'm using LangGraph to force function calls to Crossref or OpenAlex for a similar workflow. As long as you keep the flow rigid and only use the LLM for orchestration and formatting, the hallucinations pretty much disappear.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#403

Post by foxes »

sodapopcan wrote: post #362
foxes wrote: post #355

Earlier quoted context omitted.

Im really not motivated by this argument; it seems a false equivalence. Its not merely a spell checker or removing some tedium. As a professional mathematician I used wikipedia all the time to lookup quick facts before verifying it myself or elsewhere. A calculator well; I can use an actual programming language. Up until this point neither of those tools were asvertised or used by people to entirely replace human inp…

I hate to sound like a 19 year old on Reddit but: AI People: "AI is a completely unprecedented technology where its introduction is unlike the introduction of any other transformative technology in history! We must treat it totally differently!" Also AI People: "You're worried about nothing, this is just like when people were worried about the internet."

Well the internet has definitely changed things; but also it wasnt initially controlled by a bunch of megacorps with the same level of power and centralisation today.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#404

Post by mapontosevenths »

andrepd wrote: post #397
mapontosevenths wrote: post #351

Earlier quoted context omitted.

> It's a shoddy, irresponsible way to work. And also plagiarism, when you claim authorship of it. It reminds me of kids these days and their fancy calculators! Those new fangled doohickeys just aren't reliable, and the kids never realize that they won't always have a calculator on them! Everyone should just do it the good old fashioned way with slide rules! Or these darn kids and their unreliable sources like Wikiped…

> Those new fangled doohickeys just aren't reliable Except they are (unlike a chatbot, a calculator is perfectly deterministic), and the unreliability of LLMs is one of their most, if not the most , widespread target of criticism. Low effort doesn't even begin to describe your comment.

> Except they are (unlike a chatbot, a calculator is perfectly deterministic)

LLM's are supposed to be stochastic. That is not a bug, I can see why you find that disappointing but it's just the reality of the tool.

However, as I mentioned elsewhere calculators also have bugs and those bugs make their way into scientific research all the time. Floating point errors are particularly common, as are order of operations problems because physical devices get it wrong frequently and are hard to patch. Worse, they are not SUPPOSED TO BE stochastic so when they fail nobody notices until it's far too late. [0 - PDF]

Further, spreadsheets are no better, for example a scan of ~3,600 genomics papers found that about 1 in 5 had gene‑name errors (e.g., SEPT2 → “2‑Sep”) because that's how Excel likes to format things.[1] Again, this is much worse than a stochastic machine doing it's stochastic job... because it's not SUPPOSED to be random, it's just broken and on a truly massive scale.

[0] https://ttu-ir.tdl.org/server/api/core/bitstreams/7fce5b73-1...

[1]https://www.washingtonpost.com/news/wonk/wp/2016/08/26/an-al...

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#405

Post by fingerlocks »

aydyn wrote: post #401
neilv wrote: post #339

Earlier quoted context omitted.

> If these are the only errors, we are not troubled. However: we do not know if these are the only errors, they are merely a signature that the paper was submitted without being thoroughly checked for hallucinations. They are a signature that some LLM was used to generate parts of the paper and the responsible authors used this LLM without care. I am troubled by people using an LLM at all to write academic research p…

>also plagiarism To me, this is a reminder of how much of a specific minority this forum is. Nobody I know in real life, personally or at work, has expressed this belief. I have literally only ever encountered this anti-AI extremism (extremism in the non-pejorative sense) in places like reddit and here. Clearly, the authors in NeurIPS don't agree that using an LLM to help write is "plagiarism", and I would trust thei…

I find that hard to believe. Every creative professional that I know shares this sentiment. That’s several graphic designers at big tech companies, one person in print media, and one visual effects artist in the film industry. And once you include many of their professional colleagues that becomes a decent sample size.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#406

Post by sodapopcan »

foxes wrote: post #403
sodapopcan wrote: post #362

Earlier quoted context omitted.

I hate to sound like a 19 year old on Reddit but: AI People: "AI is a completely unprecedented technology where its introduction is unlike the introduction of any other transformative technology in history! We must treat it totally differently!" Also AI People: "You're worried about nothing, this is just like when people were worried about the internet."

Well the internet has definitely changed things; but also it wasnt initially controlled by a bunch of megacorps with the same level of power and centralisation today.

:pointing-up-emoji:

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#407

Post by NightBlossom »

This feels less like scientific integrity and more like predatory marketing. I find this public "shame list" approach by GPTZero deeply unethical and technically suspect for several reasons:

1. Doxxing disguised as specific criticism: Publishing the names of authors and papers without prior private notification or independent verification is not how academic corrections work. It looks like a marketing stunt to generate buzz at the expense of researchers' reputations.

2. False Positives & Methodology: How does their tool distinguish between an actual AI "hallucination" and a simple human error (e.g., a typo in a year, a broken link, or a messy BibTeX entry)? Labeling human carelessness as "AI fabrication" is libelous.

3. The "Protection Racket" Vibe: The underlying message seems to be: "Buy our tool, or next time you might be on this list." It’s creating a problem (fear of public shaming) to sell the solution.

We should be extremely skeptical of a vendor using a prestigious conference as a billboard for their product by essentially publicly shaming participants without due process.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#408

Post by jama211 »

fingerlocks wrote: post #405
aydyn wrote: post #401

Earlier quoted context omitted.

>also plagiarism To me, this is a reminder of how much of a specific minority this forum is. Nobody I know in real life, personally or at work, has expressed this belief. I have literally only ever encountered this anti-AI extremism (extremism in the non-pejorative sense) in places like reddit and here. Clearly, the authors in NeurIPS don't agree that using an LLM to help write is "plagiarism", and I would trust thei…

I find that hard to believe. Every creative professional that I know shares this sentiment. That’s several graphic designers at big tech companies, one person in print media, and one visual effects artist in the film industry. And once you include many of their professional colleagues that becomes a decent sample size.

Graphic design is a completely different kettle of fish. Comparing it to academic paper writing is disingenuous.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#409

Post by jama211 »

aydyn wrote: post #401
neilv wrote: post #339

Earlier quoted context omitted.

> If these are the only errors, we are not troubled. However: we do not know if these are the only errors, they are merely a signature that the paper was submitted without being thoroughly checked for hallucinations. They are a signature that some LLM was used to generate parts of the paper and the responsible authors used this LLM without care. I am troubled by people using an LLM at all to write academic research p…

>also plagiarism To me, this is a reminder of how much of a specific minority this forum is. Nobody I know in real life, personally or at work, has expressed this belief. I have literally only ever encountered this anti-AI extremism (extremism in the non-pejorative sense) in places like reddit and here. Clearly, the authors in NeurIPS don't agree that using an LLM to help write is "plagiarism", and I would trust thei…

Yup, and no matter how flimsy an anti-ai article is, it will skyrocket to the top of HN because of it. It makes sense though, HN users are the most likely to feel threatened by LLMs, and therefore are more likely to be anxious about them.

I don’t love ai either, but that’s the truth.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#410

Post by jama211 »

andrepd wrote: post #397
mapontosevenths wrote: post #351

Earlier quoted context omitted.

> It's a shoddy, irresponsible way to work. And also plagiarism, when you claim authorship of it. It reminds me of kids these days and their fancy calculators! Those new fangled doohickeys just aren't reliable, and the kids never realize that they won't always have a calculator on them! Everyone should just do it the good old fashioned way with slide rules! Or these darn kids and their unreliable sources like Wikiped…

> Those new fangled doohickeys just aren't reliable Except they are (unlike a chatbot, a calculator is perfectly deterministic), and the unreliability of LLMs is one of their most, if not the most , widespread target of criticism. Low effort doesn't even begin to describe your comment.

As low effort as you hand waving away any nuance because it doesn’t agree with you?
Post reply on HN