Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-fabricated scientific papers on Google Scholar

misinforeview.hks.harvard.edu

Post reply on HN
Print view View original

81–90 of 107 posts

Re: GPT-fabricated scientific papers on Google Scholar

#81

Post by __loam »

benreesman wrote: post #58
nomilk wrote: post #5

GPT might make fabricating scientific papers easier, but let's not forget how many humans fabricated scientific research in recent years - they did a great job without AI! For any who haven't seen/heard, this makes for some entertaining and eye-opening viewing! https://www.youtube.com/results?search_query=academic+fraud

I think it’s important to remember that while the tidal wave of spam just starting to crest courtesy of the less scrupulous LLM vendors is uh, necessary to address, this century’s war on epistemology was well underway already in the grand traditions of periodic wars on the idea that facts are even aspirationally, directionally worthwhile. The phrase “alternative facts” hit the mainstream in 2016 and the idea that res…

Post-modernism was a mistake.

Re: GPT-fabricated scientific papers on Google Scholar

#82

Post by __loam »

EnigmaFlare wrote: post #67
layer8 wrote: post #2

I appreciate that, appropriately, the article image is not AI-generated.

It's silly that there's a stigma attached to AI generated images in cases where it's perfectly reasonable to do. People seem to appreciate things more for the fact that they were created by spending time out of another human's life more than what it actually is.

It's built on theft and it's a negative quality signal usually.

Re: GPT-fabricated scientific papers on Google Scholar

#83

Post by benreesman »

__loam wrote: post #81
benreesman wrote: post #58

Earlier quoted context omitted.

I think it’s important to remember that while the tidal wave of spam just starting to crest courtesy of the less scrupulous LLM vendors is uh, necessary to address, this century’s war on epistemology was well underway already in the grand traditions of periodic wars on the idea that facts are even aspirationally, directionally worthwhile. The phrase “alternative facts” hit the mainstream in 2016 and the idea that res…

Post-modernism was a mistake.

Indeed. I used to think that when it hybridized with Objectivism that was the nastiest malware around but god damn if Amodei and co haven’t rootkitted society to a new level.

Re: GPT-fabricated scientific papers on Google Scholar

#84

Post by Al-Khwarizmi »

matusp wrote: post #30
Strilanc wrote: post #26

When I went to the APS March Meeting earlier this year, I talked with the editor of a scientific journal and asked them if they were worried about LLM generated papers. They said actually their main worry wasn't LLM-generated papers, it was LLM-generated reviews . LLMs are much better at plausibly summarizing content than they are at doing long sequences of reasoning, so they're much better at generating believable r…

We already got an LLM generated meta review that was very clearly just summarization of reviews. There were some pretty egregious cases of borderline hallucinated remarks. This was ACL Rolling Review, so basically the most prestigious NLP venue and the editors told us to suck it up. Very disappointing and I genuinely worry about the state of science and how this will affect people who rely on scientometric criteria.

This is a problem in general, but the unmitigated disaster that is ARR (ACL Rolling Review) doesn't help.

On the one hand, if you submit to a conference, you are forced to "volunteer" for that cycle. Which is a good idea from a "justice" point of view, but its also a sure way of generating unmotivated reviewers. Not only because a person might be unmotivated in general, but because the -rather short- reviewing period may coincide in your vacation (this happened to many people with EMNLP, whose reviewing period was in the summer) and you're not given any alternative but to "volunteer" and deal with it.

On the other hand, even regular reviewers aren't treated too well. Lately they implemented a minimum max load of 4 (which can push people towards choosing uncomfortable loads, in fact, that seems to be the purpose) and loads aren't even respected (IIRC there have been mails to the tune of "some people set a max load but we got a lot of submissions so you may get more submissions than your load, lololol").

While I don't condone using LLMs for reviewing and I would never do such a thing, I am not too surprised that these things happen given that ARR makes the already often thankless job of reviewing even more annoying.

To be honest, lately, I have gotten better quality reviews from the supposedly second-tier conferences that haven't joined ARR (e.g. this year's LREC-COLING) than from ARR. Although sample size is very small, of course.

Re: GPT-fabricated scientific papers on Google Scholar

#86

Post by RobotToaster »

oefrha wrote: post #57

How about people stop responding to titles for a change. This isn’t about papers that merely used ChatGPT and got caught by some cutting edge detection techniques, it’s about papers that blatantly include ChatGPT boilerplates like > “as of my last knowledge update” and/or “I don’t have access to real-time data” which suggests no human (don’t even need to be a researcher) read every sentence of these damn “papers”. Th…

> which suggests no human (don’t even need to be a researcher) read every sentence of these damn “papers”.

Which also suggests none of the so called reviewers or editors read the entire paper before including it in their journal...

Re: GPT-fabricated scientific papers on Google Scholar

#87

Post by ahartmetz »

matusp wrote: post #30
Strilanc wrote: post #26

When I went to the APS March Meeting earlier this year, I talked with the editor of a scientific journal and asked them if they were worried about LLM generated papers. They said actually their main worry wasn't LLM-generated papers, it was LLM-generated reviews . LLMs are much better at plausibly summarizing content than they are at doing long sequences of reasoning, so they're much better at generating believable r…

We already got an LLM generated meta review that was very clearly just summarization of reviews. There were some pretty egregious cases of borderline hallucinated remarks. This was ACL Rolling Review, so basically the most prestigious NLP venue and the editors told us to suck it up. Very disappointing and I genuinely worry about the state of science and how this will affect people who rely on scientometric criteria.

> people who rely on scientometric criteria

Not defending LLM papers at all, but these people can go to hell. If "scientometrics" was ever a good idea, after making the measure the target, it for sure isn't anymore. A longer, carefully written, comprehensive paper is rated worse than many short, incremental, hastily written papers.

Re: GPT-fabricated scientific papers on Google Scholar

#89

Post by greesil »

benreesman wrote: post #58
nomilk wrote: post #5

GPT might make fabricating scientific papers easier, but let's not forget how many humans fabricated scientific research in recent years - they did a great job without AI! For any who haven't seen/heard, this makes for some entertaining and eye-opening viewing! https://www.youtube.com/results?search_query=academic+fraud

I think it’s important to remember that while the tidal wave of spam just starting to crest courtesy of the less scrupulous LLM vendors is uh, necessary to address, this century’s war on epistemology was well underway already in the grand traditions of periodic wars on the idea that facts are even aspirationally, directionally worthwhile. The phrase “alternative facts” hit the mainstream in 2016 and the idea that res…

The metric is in fact the stock price.

Re: GPT-fabricated scientific papers on Google Scholar

#90

Post by Sakos »

Eisenstein wrote: post #66
basch wrote: post #56

Earlier quoted context omitted.

It might follow to say that current LLM;s arent trained to generate papers, BUT they also don't really need to reason. They just need to mimic the appearance of reason, follow the same pattern of progression. Ingesting enough of what amounts to executed templates will teach it to generate its own results as if output from the same template.

What is the difference between 'reasoning' and 'appearing to be reasoning' if the results are the same with the same input?

Because the results aren't the same? I use AI every day for software development and a number of other topics. It's very easy to recognize the points where the illusion breaks and how it breaks clearly indicates to me that there's no actual reasoning in the response I've gotten. It often feels like I'm doing the reasoning for the AI and not the other way around.
Post reply on HN