Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

gptzero.me

Post reply on HN
Print view View original

211–220 of 528 posts

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#211

Post by iugtmkbdfil834 »

wizzwizz4 wrote: post #191
iugtmkbdfil834 wrote: post #185

Earlier quoted context omitted.

Yeah, but doesn't publishing an easily falsifiable paper end one?

Not in most fields, unless misconduct is evident. (And what constitutes "misconduct" is cultural: if you have enough influence in a community, you can exert that influence on exactly where that definitional border lies.) Being wrong is not, and should not be , a career-ending move.

If we are aiming for quality, then being wrong absolutely should be. I would argue that is how it works in real life anyway. What we quibble over is what is the appropriate cutoff.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#212

Post by heliumtera »

azan_ wrote: post #169
heliumtera wrote: post #157

Earlier quoted context omitted.

I am sure all advancements in the history of humanity was properly peer reviewed! Including coca cola and Linux!

If you wanted to attack peer review you should've attacked peer review, not entire science. And if "muh science" was some kind of code for peer review then it's not my fault that you are awful at articulating your point. It's still not clear what the hell do you mean.

Science funding is strictly tied to publication on "respected journals" which is strictly tied to this gatekeeping horrendous process.

I won't deny I am terrible at articulating my point, but I will maintain it. We can undeniably say that science, scientific institutions, scientific periodic journals, funding and any other financial instrument constructed to promote scientific advancements is rotten by design and should be abandoned immediately. This joke serves no good.

"But what about muh scientific method?" Yeah yeah yeah, whoever thinks modern science honors logic and reason is part of the problem and has being played, and forever will be

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#213

Post by ctoth »

The innumeracy is load-bearing for the entire media ecosystem. If readers could do basic proportional reasoning, half of health journalism and most tech panic coverage would collapse overnight.

GPTZero of course knows this. "100 hallucinations across 53 papers at prestigious conference" hits different than "0.07% of citations had issues, compared to unknown baseline, in papers whose actual findings remain valid."

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#214

Post by iugtmkbdfil834 »

bnchrch wrote: post #193
iugtmkbdfil834 wrote: post #185

Earlier quoted context omitted.

Yeah, but doesn't publishing an easily falsifiable paper end one?

One, it doesnt damage your reputation as much as one would think. But two, and more importantly, no one is checking. Tree falls in the forest, no one hears, yadi-yada.

I think this is the big part of it. There is no incentive to do it even when the study can be reproduced.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#215

Post by Telaneo »

iugtmkbdfil834 wrote: post #185
parpfish wrote: post #158

Earlier quoted context omitted.

In theory, asking grad students and early career folks to run replications would be a great training tool. But the problem isn’t just funding, it’s time. Successfully running a replication doesn’t get you a publication to help your career.

Yeah, but doesn't publishing an easily falsifiable paper end one?

Not really, since nobody (for values of) ends up actually falsifying it, and if they do, it's years down the line.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#216

Post by Kinrany »

goalieca wrote: post #202
parpfish wrote: post #158

Earlier quoted context omitted.

In theory, asking grad students and early career folks to run replications would be a great training tool. But the problem isn’t just funding, it’s time. Successfully running a replication doesn’t get you a publication to help your career.

Grad students don’t get to publish a thesis on reproduction. Everyone from the undergraduate research assistant to the tenured professor with research chairs are hyper focused on “publishing” as much “positive result” on “novel” work as possible

Publishing a replication could be a prerequisite to getting the degree

The question is, how can universities coordinate to add this requirement and gain status from it

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#217

Post by mort96 »

lallysingh wrote: post #205
cogman10 wrote: post #2

Yuck, this is going to really harm scientific research. There is already a problem with papers falsifying data/samples/etc, LLMs being able to put out plausible papers is just going to make it worse. On the bright side, maybe this will get the scientific community and science journalists to finally take reproducibility more seriously. I'd love to see future reporting that instead of saying "Research finds amazing che…

On the bright side, an LLM can really help set up a reproduction environment. Perhaps repro should become the basis of peer review?

No, it can't. No LLM can purchase the equipment and chemicals and machinery you need to reproduce experiments, nor should you want it.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#218

Post by gogopromptless »

dataflow wrote: post #199
gcr wrote: post #150

Earlier quoted context omitted.

I'd personally like to see top conferences grow a "reproducibility" track. Each submission would be a short tech report that chooses some other paper to re-implement. Cap 'em at three pages, have a lightweight review process. Maybe there could be artifacts (git repositories, etc) that accompany each submission. This would especially help newer grad students learn how to begin to do this sort of research. Maybe doing…

Is it time for some sort of alternate degree to a PhD beyond a Master's? Showing, essentially, "this person can learn, implement, validate, and analyze the state of the art in this field"?

Thats what we call a Staff level engineer. Proven ability to learn, implement and validate is basically the "it factor" businesses are looking for.

If you are thinking about this from an academic angle then sure its sounds weird to say "Two Staff jobs in a row from the University of LinkedIn" as a degree. But I submit this as basically the certificate you desire.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#219

Post by LPisGood »

lxgr wrote: post #178
cogman10 wrote: post #2

Yuck, this is going to really harm scientific research. There is already a problem with papers falsifying data/samples/etc, LLMs being able to put out plausible papers is just going to make it worse. On the bright side, maybe this will get the scientific community and science journalists to finally take reproducibility more seriously. I'd love to see future reporting that instead of saying "Research finds amazing che…

> LLMs being able to put out plausible papers is just going to make it worse If correct form (LaTeX two-column formatting, quoting the right papers and authors of the year etc.) has been allowing otherwise reject-worthy papers to slip through peer review, academia arguably has bigger problems than LLMs.

Correct form and relevant citations have been, for generations up to a couple of years ago, mighty strong signals that a work is good and done by a serious and reliable author. This is no longer the case and we are worse off for it.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#220

Post by rfrey »

There's a lot of good arguments in this thread about incentives: extremely convincing about why current incentives lead to exactly this behaviour, and also why creating better incentives is a very hard problem.

If we grant that good carrots are hard to grow, what's the argument against leaning into the stick? Change university policies and processes so that getting caught fabricating data or submitting a paper with LLM hallucinations is a career ending event. Tip the expected value of unethical behaviours in favour of avoiding them. Maybe we can't change the odds of getting caught but we certainly can change the impact.

This would not be easy, but maybe it's more tractable than changing positive incentives.

Post reply on HN