Try to guess if code is real or GPT2-generated
doesnotexist.codes
Try to guess if code is real or GPT2-generated
1–10 of 108 posts
Re: Try to guess if code is real or GPT2-generated
#2I found this impressively hard at first glance. It just goes to show how difficult getting into context is in an unfamiliar codebase. I think with any amount of knowledge of anything allegedly involved (or, you know, a compiler), these examples would fall apart, but it's still an achievement.
I'm also pretty sure there are formatting, commenting, and in-string-text "tells" that indicate whether something is GPT2 reliably. Maybe I should try training an AI to figure that out...
Re: Try to guess if code is real or GPT2-generated
#3This was a fun exercise, definitely think this could be difficult to suss out for greener devs or even more experienced ones. It’d be hilarious to have this model power a live screensaver in lieu of actually being busy at times.
Re: Try to guess if code is real or GPT2-generated
#4I found this impressively hard at first glance. It just goes to show how difficult getting into context is in an unfamiliar codebase. I think with any amount of knowledge of anything allegedly involved (or, you know, a compiler), these examples would fall apart, but it's still an achievement. I'm also pretty sure there are formatting, commenting, and in-string-text "tells" that indicate whether something is GPT2 reli…
I tried using a weird indent as a signal of GPT-2... which gave me my first wrong answer. 4/5.
Re: Try to guess if code is real or GPT2-generated
#5The black background on white background makes it annoying to read.
Re: Try to guess if code is real or GPT2-generated
#6This was so much harder than I thought it was going to be. I would get a few right and then be absolutely sure of the next one and be wrong. After a while I felt like I was noticing more aesthetic differences between the gpt and real, rather than distinguishing between the two based on their content. Very interesting...
Re: Try to guess if code is real or GPT2-generated
#7For the ones that were just part of the header file, listing a bunch of instance variables and function names, it seems impossible. But for the actual code, it is possible but still quite difficult, though I spent too long in finding some logical inconsistency that gave it away.
Re: Try to guess if code is real or GPT2-generated
#8I found this impressively hard at first glance. It just goes to show how difficult getting into context is in an unfamiliar codebase. I think with any amount of knowledge of anything allegedly involved (or, you know, a compiler), these examples would fall apart, but it's still an achievement. I'm also pretty sure there are formatting, commenting, and in-string-text "tells" that indicate whether something is GPT2 reli…
I tried using a weird indent as a signal of GPT-2... which gave me my first wrong answer. 4/5.
[deleted]
Re: Try to guess if code is real or GPT2-generated
#9Got 40/50 just smashing the GPT2 button.
Re: Try to guess if code is real or GPT2-generated
#10This is difficult... because these models are just regurgitating after training on real code. Fun little site but I hope nobody reads too much into this.