I wrote before about my embarrassing time with ChatGPT during a period (https://news.ycombinator.com/item?id=44767601) - I decided to go back through those old 4o chats with 5.2 pro extended thinking, the reply was pretty funny because it first slightly ridiculed me, heh - but what it showed was: basically I would say "what 5 research papers from any area of science talk to these ideas" and it would find 1 and invent 4 if it didn't know 4 others, and not tell me, and then I'd keep working with it and it would invent what it thought might be in the papers long the way, making up new papers in it's own work to cite to make it's own work valid, lol. Anyway, I'm a moron, sure, and no real harm came of it for me, just still slightly shook I let that happen to me.
GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
141–150 of 528 posts
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#142Earlier quoted context omitted.
> I'd love to see future reporting that instead of saying "Research finds amazing chemical x which does y" you see "Researcher reproduces amazing results for chemical x which does y. First discovered by z". Most people (that I talk to, at least) in science agree that there's a reproducibility crisis. The challenge is there really isn't a good way to incentivize that work. Fundamentally (unless you're independent weal…
> The challenge is there really isn't a good way to incentivize that work. Ban publication of any research that hasn't been reproduced.
> Ban publication of any research that hasn't been reproduced.
Unless it is published, nobody will know about it and thus nobody will try to reproduce it.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#143Earlier quoted context omitted.
Nope. I am still reviewing papers that propose solutions based on a technique X, conveniently ignoring research from two years ago that shows that X cannot be used on its own. Both the paper I reviewed and the research showing X cannot be used are in the same venue!
does it seem to be legitimate ignorance or maybe folks pushing ahead regardless of x being disproved?
Poor scholarship.
However, given the feedback by other reviewers, I was the only one who knew that X doesn’t work. I am not sure how these people mark themselves as “experts” in the field if they are not following the literature themselves.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#144AI might just extinguish the entire paradigm of publish or perish. The sheer volume of papers makes it nearly impossible to properly decide which papers have merit, which are non-replicate and suspect, and which are just a desperate rush to publish. The entire practice needs to end.
Its not publish or perish so much as get grant money or perish.
Publishing is just the way to get grants.
A PI explained it to me once, something like this
Idea(s) -> Grant -> Experiments -> Data -> Paper(s) -> Publication(s) -> Idea(s) -> Grant(s)
Thats the current cycle ... remove any step and its a dead end
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#145Wow! They're literally submitting references to papers by Firstname Lastname, John Doe and Jane Smith and nobody is noticing or punishing them.
[flagged]
I'm a feyerabend sympathizer, but even he wouldn't have gone this far.
He was against establishment dogma, not pro-anti intellectualism.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#146Earlier quoted context omitted.
> I'd love to see future reporting that instead of saying "Research finds amazing chemical x which does y" you see "Researcher reproduces amazing results for chemical x which does y. First discovered by z". Most people (that I talk to, at least) in science agree that there's a reproducibility crisis. The challenge is there really isn't a good way to incentivize that work. Fundamentally (unless you're independent weal…
> The challenge is there really isn't a good way to incentivize that work. Ban publication of any research that hasn't been reproduced.
lol, how would the first paper carrying some new discovery get published?
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#147And this is the tip of the iceberg, because these are the easy to check/validate things.
I'm sure plenty of more nuanced facts are also entirely without basis.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#148Yuck, this is going to really harm scientific research. There is already a problem with papers falsifying data/samples/etc, LLMs being able to put out plausible papers is just going to make it worse. On the bright side, maybe this will get the scientific community and science journalists to finally take reproducibility more seriously. I'd love to see future reporting that instead of saying "Research finds amazing che…
In my mental model, the fundamental problem of reproducibility is that scientists have very hard time to find a penny to fund such research. No one wants to grant “hey I need $1m and 2 years to validate the paper from last year which looks suspicious”.
Until we can change how we fund science on the fundamental level; how we assign grants — it will be indeed very hard problem to deal with.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#149Yuck, this is going to really harm scientific research. There is already a problem with papers falsifying data/samples/etc, LLMs being able to put out plausible papers is just going to make it worse. On the bright side, maybe this will get the scientific community and science journalists to finally take reproducibility more seriously. I'd love to see future reporting that instead of saying "Research finds amazing che…
Maybe it will also change the whole publication as evaluation of science.
Re: GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers
#150Yuck, this is going to really harm scientific research. There is already a problem with papers falsifying data/samples/etc, LLMs being able to put out plausible papers is just going to make it worse. On the bright side, maybe this will get the scientific community and science journalists to finally take reproducibility more seriously. I'd love to see future reporting that instead of saying "Research finds amazing che…
> I'd love to see future reporting that instead of saying "Research finds amazing chemical x which does y" you see "Researcher reproduces amazing results for chemical x which does y. First discovered by z". Most people (that I talk to, at least) in science agree that there's a reproducibility crisis. The challenge is there really isn't a good way to incentivize that work. Fundamentally (unless you're independent weal…
I'd personally like to see top conferences grow a "reproducibility" track. Each submission would be a short tech report that chooses some other paper to re-implement. Cap 'em at three pages, have a lightweight review process. Maybe there could be artifacts (git repositories, etc) that accompany each submission.
This would especially help newer grad students learn how to begin to do this sort of research.
Maybe doing enough reproductions could unlock incentives. Like if you do 5 reproductions than the AC would assign your next paper double the reviewers. Or, more invasively, maybe you can't submit to the conference until you complete some reproduction.