Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-fabricated scientific papers on Google Scholar

misinforeview.hks.harvard.edu

Post reply on HN
Print view View original

101–107 of 107 posts

Re: GPT-fabricated scientific papers on Google Scholar

#101

Post by passion__desire »

kenjackson wrote: post #25
hodgesrm wrote: post #9

> Two main risks arise... First, the abundance of fabricated “studies” seeping into all areas of the research infrastructure... A second risk lies in the increased possibility that convincingly scientific-looking content was in fact deceitfully created with AI tools... A third risk: ChatGPT has no understanding of "truth" in the sense of facts reported by established, trusted sources. I'm doing a research project rel…

It sounds like your use of AI is one of the worst uses. Standard semantic search would be much better and appropriate.

Existence of LLMs make Google search even more relevant for cross-checking rather than less relevant for deep research. Daniel Dennett said we should have all levels of searches available for everyone i.e. from basic string matching to semantic matching. [0]

[0] https://youtu.be/arEvPIhOLyQ?t=1139

Re: GPT-fabricated scientific papers on Google Scholar

#102

Post by OutOfHere »

TonyTrapp wrote: post #93
OutOfHere wrote: post #53

Earlier quoted context omitted.

How do you know there is no proper proofreading? There is no way to tell, is there? Just because content was generated by an LLM doesn't in itself mean that it wasn't proofread.

> Methods > We searched and scraped Google Scholar using the Python library Scholarly (Cholewiak et al., 2023) for papers that included specific phrases known to be common responses from ChatGPT and similar applications with the same underlying model (GPT3.5 or GPT4): “as of my last knowledge update” and/or “I don’t have access to real-time data” (see Appendix A). If noone bothered to even spot and remove these, you…

IMO, at this point, AI is very necessary as a pre-reviewer to weed out such papers that haven't been proofread. This is at both the journal as well as the preprint levels, preventing them from getting an audience.

Re: GPT-fabricated scientific papers on Google Scholar

#103

Post by CatWChainsaw »

hakanderyal wrote: post #63
ceejayoz wrote: post #15

Earlier quoted context omitted.

Sure, but there are some... pretty egregious cases. https://mashable.com/article/ai-rat-penis-diagram-midjourney...

That’s the funniest piece of writing I’ve read in a longtime, thanks! I wonder what they were thinking submitting the paper.

They let the machines think for them, that's the whole problem.

Re: GPT-fabricated scientific papers on Google Scholar

#104

Post by EnigmaFlare »

layer8 wrote: post #94
EnigmaFlare wrote: post #67

Earlier quoted context omitted.

It's silly that there's a stigma attached to AI generated images in cases where it's perfectly reasonable to do. People seem to appreciate things more for the fact that they were created by spending time out of another human's life more than what it actually is.

It would be silly if they were indistinguishable from human-created images, but they aren't, exhibiting the typical AI artifacts and weirdness, and thereby signal a lack of care/caring.

"lack of care" - that's the part about spending time out of another human's life. It's not the poor quality that's the problem but the lack of human effort. Oil paintings are full of visible brush strokes which are an artifact but people love them. For most applications of art - advertising, background decorations, news article pictures, etc. there really is no need to show that humans spent effort on it.

The human effort idea is even a bit morally objectionable. You can feel that you're worth more than others because more of the lives of others were consumed to create your possessions. It's a zero sum game where poor people can never afford high-care art because their time is worth less than the artist's.

Re: GPT-fabricated scientific papers on Google Scholar

#106

Post by xandrius »

daghamm wrote: post #78

Last time we discussed this, someone basically searched for phrases such as "certainly I can do X for you" and assumed that meant GPT was used. HN noticed that many of the accused papers actually predated openai. Hope this research is better.

How else would that phrase go into a real paper then?

Re: GPT-fabricated scientific papers on Google Scholar

#107

Post by krisrs »

refibrillator wrote: post #23

Hmm there may be a bug in the authors’ python script that searches google scholar for the phrases "as of my last knowledge update" or "I don't have access to real-time data". You can see the code in appendix B. The bug happens if the ‘bib’ key doesn’t exist in the api response. That leads to the urls array having more rows than the paper_data array. So the columns could become mismatched in the final data frame. It s…

Hi there! My name is Kristofer, one of the authors of this research note. I also wrote the script. We were notified via email about this comment. Please see below for our response. Thank you for your interest in our research! (I'm removing the sender's name to respect their privacy)

""" Dear XXXX,

MY name is Kristofer, I’m one of the co-authors for the GPT paper. I also wrote the script for the data collection. Jutta forwarded your email regarding the possible bug.

First of all, let me apologise for the late response. Apparently your email made its way to the spam folder, which of course is regrettable. I would also like to thank you for reaching out to us. We are pleased to see the interest of the HN community in transparent and reliable research.

We looked at the comment and the concern around the bug. We’d like to point out that the original commenter was right in saying “it does seem like something they would catch when manually reviewing the papers”. We in fact reviewed the output manually and carefully for any potential errors. In other words, we opened and searched for the query string manually, which also helped determine whether the use of LLMs was declared in some form or other. This is of course a sensitive topic and we took great care to be thorough.

Nevertheless, we once more did a manual review of the code and the data, in light of this potential bug, and we’re glad to say no row-column mismatch is present. You can find the data here: https://doi.org/10.7910/DVN/WUVD8X

Please don’t hesitate if you have any more questions.

All the best, Kristofer """

Post reply on HN