Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

gptzero.me

Post reply on HN
Print view View original

321–330 of 528 posts

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#321

Post by pas »

jklinger410 wrote: post #283
paulmist wrote: post #201

Earlier quoted context omitted.

> When your entire job is confirming that science is valid, I expect a little more humility when it turns out you've missed a critical aspect. I wouldn't call a misformed reference a critical issue, it happens. That's why we have peer reviews. I would contend drawing superficially valid conclusions from studies through use of AI is a much more burning problem that speaks more to the integrity of the author. > It will…

> I wouldn't call a misformed reference a critical issue, it happens. That's why we have peer reviews. Crazy to say this in a discussion where peer review missed hallucinated citations

even the fact that citations are not automatically verified by the journal is crazy, the whole academia and publishing enterprise is an empire built on inefficiency, hubris, and politics (but I'm repeating myself).

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#322

Post by derektank »

bonoboTP wrote: post #297
Kinrany wrote: post #216

Earlier quoted context omitted.

Publishing a replication could be a prerequisite to getting the degree The question is, how can universities coordinate to add this requirement and gain status from it

Prerequisite required by who, and why is that entity motivated to design such a requirement? Universities also want more novel breakthrough papers to boast about and to outshine other universities in the rankings. And if one is honest, other researchers also get more excited about new ideas than a failed replication that may for a thousand different reasons and the original authors will argue you did something wrong,…

> Prerequisite required by who, and why is that entity motivated to design such a requirement?

Grant awarding institutions like the NIH and NSF presumably? The NSF has as one of its functions, “to develop and encourage the pursuit of a national policy for the promotion of basic research and education in the sciences”. Encouraging the replication of research as part of graduate degree curricula seems to fall within bounds. And the government’s interest in science isn’t novelty per se, it’s the creation and dissemination of factually correct information that can be useful to its constituents.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#323

Post by dataflow »

gcr wrote: post #271
dataflow wrote: post #199

Earlier quoted context omitted.

Is it time for some sort of alternate degree to a PhD beyond a Master's? Showing, essentially, "this person can learn, implement, validate, and analyze the state of the art in this field"?

That sounds precisely like the function of a Ph.D. to me.

A PhD is for making new contributions to a field, not validating existing ones.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#325

Post by Der_Einzige »

currymj wrote: post #236

Especially for your first NeurIPS paper as a PhD student, getting one published is extremely lucrative. Most big tech PhD intern job postings have NeurIPS/ICML/ICLR/etc. first author paper as a de facto requirement to be considered. It's like getting your SAG card. If you get one of these internships, it effectively doubles or triples your salary that year right away. You will make more in that summer than your PhD s…

This is 100% true, if anything you’re massively undercounting the value of publications.

Most industry AI jobs that aren’t research based know that NeurIPS publications are a huge deal. Many of the managers don’t even know what a workshop is (so you can pass off NeurIPS workshop work as just “NeurIPS”)

A single first author main conference work effectively allows a non Ph.D holder to be treated like they have a Ph.d (be qualified for professional researcher jobs). This means that a decent engineer with 1 NeurIPS publication is easily worth 300K+ YOY assuming US citizen. Even if all they have is a BS ;)

And if you are lucky to get a spotlight or an oral, that’s probably worth closer to 7 figures…

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#326

Post by bonoboTP »

derektank wrote: post #322
bonoboTP wrote: post #297

Earlier quoted context omitted.

Prerequisite required by who, and why is that entity motivated to design such a requirement? Universities also want more novel breakthrough papers to boast about and to outshine other universities in the rankings. And if one is honest, other researchers also get more excited about new ideas than a failed replication that may for a thousand different reasons and the original authors will argue you did something wrong,…

> Prerequisite required by who, and why is that entity motivated to design such a requirement? Grant awarding institutions like the NIH and NSF presumably? The NSF has as one of its functions, “to develop and encourage the pursuit of a national policy for the promotion of basic research and education in the sciences”. Encouraging the replication of research as part of graduate degree curricula seems to fall within bo…

The commenter I was replying to wanted it to be a prerequisite for a degree, not for a grant. Grant awarding institutions also have to justify their spending to other parts of government and/or parliament (specifically, politicians). Both politicians and the public want to see breakthrough results that have the potential to cure cancer and whatnot. They want to boast that their funding contributed to winning some big-name prize and so on. You have to think with the mind of specific people in specific positions and what makes them look good, what gets them praise, promotions and friends.

> And the government’s interest in science isn’t novelty per se, it’s the creation and dissemination of factually correct information that can be useful to its constituents.

This sounds very naive.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#327

Post by dataflow »

gogopromptless wrote: post #218
dataflow wrote: post #199

Earlier quoted context omitted.

Is it time for some sort of alternate degree to a PhD beyond a Master's? Showing, essentially, "this person can learn, implement, validate, and analyze the state of the art in this field"?

Thats what we call a Staff level engineer. Proven ability to learn, implement and validate is basically the "it factor" businesses are looking for. If you are thinking about this from an academic angle then sure its sounds weird to say "Two Staff jobs in a row from the University of LinkedIn" as a degree. But I submit this as basically the certificate you desire.

No, this is not at all being a staff engineer. One is about delivering high-impact projects toward a business's needs, with all the soft/political things that involves, and the other is about implementing and validating cutting-edge research, with all the deep academic and technical knowledge and work that that involves. They're incredibly different skillsets, and many people doing one would easily fail in the other.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#328

Post by abalone »

At least in one case the authors claimed to use ChatGPT to "generate the citations after giving it author-year in-text citations, titles, or their paraphrases." They pasted the hallucinations in without checking. They've since responded with corrections to real papers that in most cases are very similar to the hallucination, lending credibility to their claim.[1]

Not great, but to be clear this is different from fabricating the whole paper or the authors inventing the citations. (In this case at least.)

[1] https://openreview.net/forum?id=IiEtQPGVyV

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#329

Post by Der_Einzige »

Proziam wrote: post #15
CGMthrowaway wrote: post #9

Which is worse: a) p-hacking and suppressing null results b) hallucinations c) falsifying data Would be cool to see an analysis of this

All 3 of these should be categorized as fraud, and punished criminally.

Only when we can arrest people who say dumb stuff on the internet too. Much like how trump and bubba (bill Clinton) should share a jail cell, those who pontificate about what they don’t know about (I.e non academics critiquing academia) can sit in the same jail cell as the supposed criminal academics.

You gotta horse trade if you want to win. Take one for the team or get out of the way.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#330

Post by rovr138 »

SaaSasaurus wrote: post #324

I'm surprised it's only 100, honestly. Also feels a little sensationalized... Before AI I wonder how many "hallucinations" were in human-written papers. Is there any data on this?

These are 100, already reviewed papers and accepted papers, by researchers in their areas of expertise. Usually PhD's and Professors.... They judge.

These are not all the submissions that they received. The review process can be... brutal for some people (depending on the quality of their submission)

Post reply on HN