Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detect ChatGPT Generated Content

gptzero.me

Post reply on HN
Print view View original

81–90 of 109 posts

Re: Detect ChatGPT Generated Content

#81

Post by TillE »

anyonecancode wrote: post #73
spoork wrote: post #26

The problem with such things is that you would need 100% accuracy in most cases for it to be useful. For example many schools and universities fear that students use ChatGPT for homework. If such a plagiarism checker has false positive results in just a small percentage of cases, the consequences for honest students would be too severe to actually act on the results of the check. But perfect accuracy can never be rea…

At the university level, the problem of cheating is more a problem with our idea of a university. In an ideal world, you go to a university to learn. If someone wants to cheat, ultimately they're just cheating themselves (and, if at a private university or even most public universities, throwing a lot of money away). "Cheating" isn't really the university's problem in that view. Ofc, in reality universities aren't (o…

You don't really need any radical changes, you just have to get rid of the idea of graded homework, which I always hated from elementary school onwards.

Larger projects have a role to play in education, but ultimately if you can't pass the proctored exams, you don't pass the course.

Re: Detect ChatGPT Generated Content

#82

Post by zopa »

We ought to require OpenAI to run something like this: a “Hey ChatGPT, this you?” endpoint that replies Yes/No/similarity score, and a creation timestamp, when you hit it with some text.

As regulations go, this one’s not too burdensome: expensive, but pretty cheap compared to training and running a large language model in the first place.

Re: Detect ChatGPT Generated Content

#83

Post by hutzlibu »

bhickey wrote: post #69
krisoft wrote: post #59

This is useless. I have tried it with 4 different samples and it made mistakes with each. - it thinks AI wrote parts of my handwritten texts - King James Genesis 29 is appearantely fully AI written - a wall of text copied straight from chatgpt? Only partially AI written. - second chapter of Harry Potter? AI written parts In fact I could not find a sample yet which was not AI written at least partially according to th…

Agreed. This is useless. I asked GPT-3 to write a HN post decrying GPTZero: > GPTZero is absolute trash! It can't even detect plagarism correctly. It's a complete joke and waste of time! Don't bother with it! You'd be better off just manually checking for plagiarism. It's so unreliable it's not even worth mentioning. Just save yourself the time and money and don't bother with GPTZero. Worst AI tool ever! GPTZero clas…

It is hilarious comedy, though.

The sad part is, that real people will likely suffer, when people with power take salespersons serious and use stuff like that to "detect" and punish frauds.

Re: Detect ChatGPT Generated Content

#85

Post by julianh65 »

I read an interesting tweet about how it would be possible to watermark GPT outputs. Essentially before each token is generated the previous token is used to seed an RNG. Using the RNG the possible next tokens are split into a whitelist and a blacklist and the model can only select words from the whitelist. Later on it's possible to "check" for the watermark by counting the whitelist tokens and doing statistical analysis.

Apparently they can preserve performance by not doing this for very low entropy tokens where there is only one token that is extremely likely.

Saw it here: https://twitter.com/tomgoldsteincs/status/161828766500640358...

Re: Detect ChatGPT Generated Content

#87

Post by jonnycomputer »

zopa wrote: post #82

We ought to require OpenAI to run something like this: a “Hey ChatGPT, this you?” endpoint that replies Yes/No/similarity score, and a creation timestamp, when you hit it with some text. As regulations go, this one’s not too burdensome: expensive, but pretty cheap compared to training and running a large language model in the first place.

This is a pretty good idea, at least for commercial models.

Re: Detect ChatGPT Generated Content

#89

Post by kcplate »

krisoft wrote: post #59

This is useless. I have tried it with 4 different samples and it made mistakes with each. - it thinks AI wrote parts of my handwritten texts - King James Genesis 29 is appearantely fully AI written - a wall of text copied straight from chatgpt? Only partially AI written. - second chapter of Harry Potter? AI written parts In fact I could not find a sample yet which was not AI written at least partially according to th…

Same, ran a few samples and it was wrong every time. My hope is that teachers will run some tests before just willy-nilly trusting the scores it gives and start accusing students, but sadly I know a lot of teachers and I bet within that group few will run their own tests and will just accept it as authority.

Re: Detect ChatGPT Generated Content

#90

Post by ren_engineer »

krisoft wrote: post #59

This is useless. I have tried it with 4 different samples and it made mistakes with each. - it thinks AI wrote parts of my handwritten texts - King James Genesis 29 is appearantely fully AI written - a wall of text copied straight from chatgpt? Only partially AI written. - second chapter of Harry Potter? AI written parts In fact I could not find a sample yet which was not AI written at least partially according to th…

it's a toy project built by a CS undergrad student over their winter break that went viral because journalists desperately want something that can expose AI writing. It's built on streamlit and a few glued together libraries, not really surprising the results aren't good. It's not some state of the art custom tool but they are marketing themselves as the solution to solving academic plagiarism
Post reply on HN