Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

gptzero.me

Post reply on HN
Print view View original

191–200 of 528 posts

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#191

Post by wizzwizz4 »

iugtmkbdfil834 wrote: post #185
parpfish wrote: post #158

Earlier quoted context omitted.

In theory, asking grad students and early career folks to run replications would be a great training tool. But the problem isn’t just funding, it’s time. Successfully running a replication doesn’t get you a publication to help your career.

Yeah, but doesn't publishing an easily falsifiable paper end one?

Not in most fields, unless misconduct is evident. (And what constitutes "misconduct" is cultural: if you have enough influence in a community, you can exert that influence on exactly where that definitional border lies.) Being wrong is not, and should not be, a career-ending move.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#192

Post by parpfish »

iugtmkbdfil834 wrote: post #185
parpfish wrote: post #158

Earlier quoted context omitted.

In theory, asking grad students and early career folks to run replications would be a great training tool. But the problem isn’t just funding, it’s time. Successfully running a replication doesn’t get you a publication to help your career.

Yeah, but doesn't publishing an easily falsifiable paper end one?

But the thing is… nobody is doing the replication to falsify it. And if the did, it wouldn’t be published because it’s a null result

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#193

Post by bnchrch »

iugtmkbdfil834 wrote: post #185
parpfish wrote: post #158

Earlier quoted context omitted.

In theory, asking grad students and early career folks to run replications would be a great training tool. But the problem isn’t just funding, it’s time. Successfully running a replication doesn’t get you a publication to help your career.

Yeah, but doesn't publishing an easily falsifiable paper end one?

One, it doesnt damage your reputation as much as one would think.

But two, and more importantly, no one is checking.

Tree falls in the forest, no one hears, yadi-yada.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#194

Post by Proziam »

WarmWash wrote: post #103
Proziam wrote: post #80

Earlier quoted context omitted.

Stealing more than a few thousand dollars is a felony, and felonies are handled in criminal court, not civil. EDIT - The threshold amount varies. Sometimes it's as low as a few hundred dollars. However, the point stands on its own, because there's no universe where the sum in question is in misdemeanor territory.

It would fall under the domain of contract law, because maybe the contract of the grant doesn't prohibit what the researcher did. The way to determine that would be in court - civil court. Most institutions aren't very chill with grant money being misused, so we already don't need to burden then state with getting Johnny muncipal prosecutor to try and figure out if gamma crystallization imaging sources were incorrect…

Fraud implies intent, either intent to deceive or intentionally negligent.

If you're taking public funds (directly or otherwise) with the intent to either:

A) Do little to no real work, and pass of the work of an AI as being your own work, or

B) Knowingly publish falsified data

Then you are, without a single shred of doubt, in criminal fraud territory. Further, the structural damage you inflict when you do the above is orders of magnitude greater than the initial fraud itself. That is a matter for civil courts ("Our company based on development on X fraudulent data, it cost us Y in damages").

Whether or not charges are pressed is going to happen way after all the internal reviews have demonstrated the person being charged has gone beyond the "honest mistake" threshold. It's like Walmart not bothering to call the cops until you're into felony territory, there's no point in doing so.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#195

Post by azan_ »

gcr wrote: post #150
StableAlkyne wrote: post #82

Earlier quoted context omitted.

> I'd love to see future reporting that instead of saying "Research finds amazing chemical x which does y" you see "Researcher reproduces amazing results for chemical x which does y. First discovered by z". Most people (that I talk to, at least) in science agree that there's a reproducibility crisis. The challenge is there really isn't a good way to incentivize that work. Fundamentally (unless you're independent weal…

I'd personally like to see top conferences grow a "reproducibility" track. Each submission would be a short tech report that chooses some other paper to re-implement. Cap 'em at three pages, have a lightweight review process. Maybe there could be artifacts (git repositories, etc) that accompany each submission. This would especially help newer grad students learn how to begin to do this sort of research. Maybe doing…

The problem is that reproducing something is really, really hard! Even if something doesn't reproduce in one experiment, it might be due to slight changes in some variables we don't even think about. There are some ways to circumvent it (e.g. team that's being reproduced cooperating with reproducing team and agreeing on what variables are important for the experiemnt and which are not), but it's really hard. The solutions you propose will unfortunately incentivize bad reproductions and we might reject theories that are actually true because of that. I think that one of the best way to fight the crisis is to actually improve quality of science - articles where authors reject to share their data should be automatically rejected. We should also move towards requiring preregistration with strict protocols for almost all studies.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#196

Post by zipy124 »

gcr wrote: post #120
jacquesm wrote: post #99

Earlier quoted context omitted.

But that actually strengthens those citiations. The I scratch your back you scratch mine ones are the ones I'm getting at and that is quite hard to do with old and wonderful stuff, the authors there are probably not in a position to reciprocate by virtue of observing the grass from the other side.

I think it's a hard problem. The semanticscholar folks are doing the sort of work that would allow them to track this; I wonder if they've thought about it. A somewhat-related parable: I once worked in a larger lab with several subteams submitting to the same conference. Sometimes the work we did was related, so we both cited each other's paper which was also under review at the same venue. (These were flavor citatio…

The flavour citations in related work is the best place to launder citations.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#198

Post by wizzwizz4 »

sroussey wrote: post #184
MetaWhirledPeas wrote: post #160

Earlier quoted context omitted.

> Eventually you may arrive at something like the H-index, which is defined as "The highest number H you can pick, where H is the number of papers you have written with H citations." It's the Google search algorithm all over again. And it's the certificate trust hierarchy all over again. We keep working on the same problems. Like the two cases I mentioned, this is a matter of making adjustments until you have the des…

Incentives. First X people that reproduce Y get Z percent of patent revenue. Or something similar.

I'm delighted to inform you that I have reproduced every patent-worthy finding of every major research group active in my field in the past 10 years. You can check my data, which is exactly as theory predicts (subject to some noise consistent with experimental error). I accept payment in cash.

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#199

Post by dataflow »

gcr wrote: post #150
StableAlkyne wrote: post #82

Earlier quoted context omitted.

> I'd love to see future reporting that instead of saying "Research finds amazing chemical x which does y" you see "Researcher reproduces amazing results for chemical x which does y. First discovered by z". Most people (that I talk to, at least) in science agree that there's a reproducibility crisis. The challenge is there really isn't a good way to incentivize that work. Fundamentally (unless you're independent weal…

I'd personally like to see top conferences grow a "reproducibility" track. Each submission would be a short tech report that chooses some other paper to re-implement. Cap 'em at three pages, have a lightweight review process. Maybe there could be artifacts (git repositories, etc) that accompany each submission. This would especially help newer grad students learn how to begin to do this sort of research. Maybe doing…

Is it time for some sort of alternate degree to a PhD beyond a Master's? Showing, essentially, "this person can learn, implement, validate, and analyze the state of the art in this field"?

Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

#200

Post by Sparkyte »

cogman10 wrote: post #2

Yuck, this is going to really harm scientific research. There is already a problem with papers falsifying data/samples/etc, LLMs being able to put out plausible papers is just going to make it worse. On the bright side, maybe this will get the scientific community and science journalists to finally take reproducibility more seriously. I'd love to see future reporting that instead of saying "Research finds amazing che…

If there is one thing which scientific reports must require is not using AI to produce the documentation. They can be of the data but not of the source or anything else. AI is a tool, not a replacement for actual work.
Post reply on HN