Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detect ChatGPT Generated Content

gptzero.me

Post reply on HN
Print view View original

51–60 of 109 posts

Re: Detect ChatGPT Generated Content

#51

Post by another-dave »

101008 wrote: post #18

I tried with a few texts that ChatGPT generated for me and it says it was entirely generated by a human. It's true, I fixed some things here and there, for cohesion basically, but I think it's almost impossible for these tools to get it right.

Conversely, I wrote a few sentences in simple English off the top of my head (about The Sound of Silence, because they were talking about it on the radio this morning) and it said it was "likely to be written entirely by AI".

False positives in a plagiarism tool are pretty bad IMO. It should definitely skew towards "we can't be certain" rather than "definitely AI".

Re: Detect ChatGPT Generated Content

#53

Post by reset-password »

I thought to make something similar but what scared me away was the idea that some educational institution could use it and decide a student was being dishonest due to a false positive. I wouldn't want to bear responsibility for something like that.

And secondly, this is a never-ending race. Even if it were to be able to detect ChatGPT content with 100% accuracy today, it would just be used to assist in training another model to defeat it.

Re: Detect ChatGPT Generated Content

#54

Post by pwdisswordfishc »

falcor84 wrote: post #37
sorokod wrote: post #5

Expect an escalating arms race between models attempting to disguise their output and tools attempting to detect generated content. There may also be escalating social and perhaps legal penalties too.

I'll just put this xkcd here - https://xkcd.com/810/ (Constructive)

This doesn’t work. Stack Exchange tried it, and it turns out there is a lot of misleading, superficial content that easily gets a high number of upvotes from newbies who don’t know better.

And that is without even considering bots.

Re: Detect ChatGPT Generated Content

#55

Post by ArcaneMoose »

What I think would be interesting is to add the assignment / questions as input for the tool to consider in its evaluation. That way your tool could plug these into ChatGPT, ask it to reword it several ways (mimicking the likely path a plagiarizer might take to actually generate content) and do some diff/comparison as part of the evaluation.

Even so, I agree with most of the comments that it would be exceeding difficult to truly identify this stuff since you can ask ChatGPT to reword things in very specific ways or just edit it yourself (aside from blatant false positives)

Re: Detect ChatGPT Generated Content

#56

Post by sorokod »

pwdisswordfishc wrote: post #54
falcor84 wrote: post #37

Earlier quoted context omitted.

I'll just put this xkcd here - https://xkcd.com/810/ (Constructive)

This doesn’t work. Stack Exchange tried it, and it turns out there is a lot of misleading, superficial content that easily gets a high number of upvotes from newbies who don’t know better. And that is without even considering bots.

Yeah, war is hell.

Re: Detect ChatGPT Generated Content

#58

Post by pigtailgirl »

-- I copy and pasted GPT3 reply to - what are Jim Crow Laws? - said written by AI - then copy paste wikipeda first paragraph - said written by AI - then I wrote it myself - said written by AI - then I copy/paste 1 word over and over and over 100 or so times - it says "likely written by human but some by AI" then - highlights the whole text and notes - likely written by AI - hahhaa --

Re: Detect ChatGPT Generated Content

#59

Post by krisoft »

This is useless. I have tried it with 4 different samples and it made mistakes with each.

- it thinks AI wrote parts of my handwritten texts

- King James Genesis 29 is appearantely fully AI written

- a wall of text copied straight from chatgpt? Only partially AI written.

- second chapter of Harry Potter? AI written parts

In fact I could not find a sample yet which was not AI written at least partially according to this.

Re: Detect ChatGPT Generated Content

#60

Post by _just7_ »

krisoft wrote: post #59

This is useless. I have tried it with 4 different samples and it made mistakes with each. - it thinks AI wrote parts of my handwritten texts - King James Genesis 29 is appearantely fully AI written - a wall of text copied straight from chatgpt? Only partially AI written. - second chapter of Harry Potter? AI written parts In fact I could not find a sample yet which was not AI written at least partially according to th…

The only way I could get it to say something was completely written by a human, was by adding spelling mistakes
Post reply on HN