[flagged]
GPT-fabricated scientific papers on Google Scholar
21–30 of 107 posts
Re: GPT-fabricated scientific papers on Google Scholar
#22Honestly what we need to do is establish much stronger credentialing schemes. The "only a good guy with an AI can stop a bad guy with an AI" approach of trying to filter out bad content is just a hopeless arms race and unproductive.
In a sense we need to go back two steps and websites need to be much stronger curators of knowledge again, and we need some reliable ways to sign and attribute real authorship to publications. So that when someone publishes a fake paper there is always a human being who signed it and can be held accountable. There's a practically unlimited number of automated systems, but only a limited number of people trying to benefit from it.
In the same way https went from being rare to being the norm because the assumption that things are default-authentic doesn't hold, the same just needs to happen to publishing. If you have a functioning reputation system and you can put on a price on fake information 99% of it is dis-incentivized.
Re: GPT-fabricated scientific papers on Google Scholar
#23Hmm there may be a bug in the authors’ python script that searches google scholar for the phrases "as of my last knowledge update" or "I don't have access to real-time data". You can see the code in appendix B.
The bug happens if the ‘bib’ key doesn’t exist in the api response. That leads to the urls array having more rows than the paper_data array. So the columns could become mismatched in the final data frame. It seems they made a third array called flag which could be used to detect and remove the bad results, but it’s not used any where in the posted code.
Not clear to me how this would affect their analysis, it does seem like something they would catch when manually reviewing the papers. But perhaps the bibliographic data wasn’t reviewed and only used to calculate the summary stats etc.
Re: GPT-fabricated scientific papers on Google Scholar
#24Just because ChatGPT was used to help write a paper doesn't in itself mean that the data or findings are fabricated.
How can I trust the paper when there is no proper proofreading?
Re: GPT-fabricated scientific papers on Google Scholar
#25> Two main risks arise... First, the abundance of fabricated “studies” seeping into all areas of the research infrastructure... A second risk lies in the increased possibility that convincingly scientific-looking content was in fact deceitfully created with AI tools... A third risk: ChatGPT has no understanding of "truth" in the sense of facts reported by established, trusted sources. I'm doing a research project rel…
It sounds like your use of AI is one of the worst uses. Standard semantic search would be much better and appropriate.
Re: GPT-fabricated scientific papers on Google Scholar
#26When I went to the APS March Meeting earlier this year, I talked with the editor of a scientific journal and asked them if they were worried about LLM generated papers. They said actually their main worry wasn't LLM-generated papers, it was LLM-generated reviews.
LLMs are much better at plausibly summarizing content than they are at doing long sequences of reasoning, so they're much better at generating believable reviews than believable papers. Plus reviews are pretty tedious to do, giving an incentive to half-ass it with an LLM. Plus reviews are usually not shared publicly, taking away some of the potential embarrassment.
Re: GPT-fabricated scientific papers on Google Scholar
#27> Two main risks arise... First, the abundance of fabricated “studies” seeping into all areas of the research infrastructure... A second risk lies in the increased possibility that convincingly scientific-looking content was in fact deceitfully created with AI tools... A third risk: ChatGPT has no understanding of "truth" in the sense of facts reported by established, trusted sources. I'm doing a research project rel…
[deleted]
Re: GPT-fabricated scientific papers on Google Scholar
#28GPT might make fabricating scientific papers easier, but let's not forget how many humans fabricated scientific research in recent years - they did a great job without AI! For any who haven't seen/heard, this makes for some entertaining and eye-opening viewing! https://www.youtube.com/results?search_query=academic+fraud
Is there good data on how many are fraudulent? I know there’s reasonable data on replicability issues, but that’s potentially different.
Re: GPT-fabricated scientific papers on Google Scholar
#29[dead]
Re: GPT-fabricated scientific papers on Google Scholar
#30When I went to the APS March Meeting earlier this year, I talked with the editor of a scientific journal and asked them if they were worried about LLM generated papers. They said actually their main worry wasn't LLM-generated papers, it was LLM-generated reviews . LLMs are much better at plausibly summarizing content than they are at doing long sequences of reasoning, so they're much better at generating believable r…
We already got an LLM generated meta review that was very clearly just summarization of reviews. There were some pretty egregious cases of borderline hallucinated remarks. This was ACL Rolling Review, so basically the most prestigious NLP venue and the editors told us to suck it up. Very disappointing and I genuinely worry about the state of science and how this will affect people who rely on scientometric criteria.