Enigma: GPT-2 trained on 10K Nature Papers: Can you spot the difference?
1–10 of 107 posts
Re: Enigma: GPT-2 trained on 10K Nature Papers: Can you spot the difference?
#2Now here is an application where the GPT stuff can really shine, which is trying to convince people that aren't domain experts that something is speaking from authority, even if the reader doesn't intend to get anything meaningful from the material either way.
Re: Enigma: GPT-2 trained on 10K Nature Papers: Can you spot the difference?
#3On easy, 7 correct and 0 wrong was enough to tell me that yes, I can (I have been reading Nature for years though).
Re: Enigma: GPT-2 trained on 10K Nature Papers: Can you spot the difference?
#4STEM people love to bring up the sokal affair. the same STEM people also don't realize that many journals and conferences in STEM have been tricked by things like this (more specifically precursors using HMMs and etc).
https://en.wikipedia.org/wiki/List_of_scholarly_publishing_s...
edit: don't understand why i'm getting downvoted. is my comment not relevant to a post about the plausibility of abstracts generated by ML models?
Re: Enigma: GPT-2 trained on 10K Nature Papers: Can you spot the difference?
#5I'm curious if the trained model is available. It would be very fun to play with.
Re: Enigma: GPT-2 trained on 10K Nature Papers: Can you spot the difference?
#6Nice advanced logic riddles.
Re: Enigma: GPT-2 trained on 10K Nature Papers: Can you spot the difference?
#7It seems some of the fake ones could easily have been real (e.g. the one about the 3d structure of bound Ach receptor) . I guess the brevity of the text helps to make it make sense and to make it indistinguishable
Re: Enigma: GPT-2 trained on 10K Nature Papers: Can you spot the difference?
#8On easy, 7 correct and 0 wrong was enough to tell me that yes, I can (I have been reading Nature for years though).
On easy, ten correct and zero wrong was enough for me to realize the same (I have never read Nature).
Trying hard right now, will report results after.
Edit: Yeah, it's the same deal. Length becomes less of a giveaway but its errors become more obvious.
Re: Enigma: GPT-2 trained on 10K Nature Papers: Can you spot the difference?
#9Easy mode is cake.
Hard mode is good enough that I'd like to see some sort of distance metric to the nearest real story, to be sure the model isn't accidentally copying truth.
Re: Enigma: GPT-2 trained on 10K Nature Papers: Can you spot the difference?
#10The sad thing is that often there's an equal mental effort to read GPT articles and the real ones. It's as if people are trying to make their papers as incomprehensible as possible.