[deleted]
Detect ChatGPT Generated Content
61–70 of 109 posts
Re: Detect ChatGPT Generated Content
#62Is there a chatgpt and human text dataset similar to gpt2 released by openAI?
Re: Detect ChatGPT Generated Content
#63This is useless. I have tried it with 4 different samples and it made mistakes with each. - it thinks AI wrote parts of my handwritten texts - King James Genesis 29 is appearantely fully AI written - a wall of text copied straight from chatgpt? Only partially AI written. - second chapter of Harry Potter? AI written parts In fact I could not find a sample yet which was not AI written at least partially according to th…
The GNU preamble is entirely AI written. According to this tool. To be honest I always suspected Stallman. :)
Re: Detect ChatGPT Generated Content
#64The problem with such things is that you would need 100% accuracy in most cases for it to be useful. For example many schools and universities fear that students use ChatGPT for homework. If such a plagiarism checker has false positive results in just a small percentage of cases, the consequences for honest students would be too severe to actually act on the results of the check. But perfect accuracy can never be rea…
> The problem with such things is that you would need 100% accuracy in most cases for it to be useful. Why? If there is 75% confidence a students report was generated using ChatGPT, then that’s enough to sit down with the student and discuss the content in person and see if they actually know it. A tool such as this could help the teacher having to avoid doing this with every student, and also reinforce to students t…
This is such a short-sighted view. Students who wish to cheat using ChatGPT will immediately start running their text through these tools preemptively, changing some phrases here and there and add spelling errors to ensure they don't get caught. You're left with innocent students being accused of cheating on regular basis.
Maybe the education system will finally learn that asking students to merely recite information that can be found by anyone, anywhere, in less than 10 seconds isn't helpful in judging their understanding of the subject, except in very limited scenarios.
Re: Detect ChatGPT Generated Content
#65The problem with such things is that you would need 100% accuracy in most cases for it to be useful. For example many schools and universities fear that students use ChatGPT for homework. If such a plagiarism checker has false positive results in just a small percentage of cases, the consequences for honest students would be too severe to actually act on the results of the check. But perfect accuracy can never be rea…
This seems to be a reductive response to a confidence based policy approach. There are many systems out there today that provide a confidence score of bots vs humans. I happen to work with a product that uses this exact approach and it's highly effective. So with education scenarios I think that the teacher/professor/administration will use these systems to inspect the entirety of the submissions. From there a baseline value will be derived and from there outliers will appear that may require deeper analysis/interview. Schools are in the position that they can't get it wrong more often than not (they need tuition dollars after all), but a severe detractor will need to be hanging over the heads of students to avoid cheating with it.
I think the other thing we'll see is a nanny state approach by some educational institutions. Falling in a trap of being sold on software to "block" or "monitor" students using these tools. It would be easy to implement on a campus network to a certain extent (correlating student network login to URL accessed and potentially MitM) but the reality is smart students will know better and will use phone hotspots and VPN. The other dark side to consider is that the owners of ChatGPT could provide logs of user accounts and queries to higher education as a service.
At the end of the day my guess is all approaches are going to be tested at some level. But the cat is out of the bag and this is going to generate some very interesting countermeasure solutions/approaches along the way.
Re: Detect ChatGPT Generated Content
#66I wonder if there is a way to watermark generated text, not by adding extra spaces or using invisible characters but by using grammar. Example naive rule “every 13th word MUST be a proposition”. Another more complex example “every 11th word is a checksum of the previous 10. You encode a few bits in every word i.e noun=1 verb=2 article=3 etc”
Re: Detect ChatGPT Generated Content
#67This is useless. I have tried it with 4 different samples and it made mistakes with each. - it thinks AI wrote parts of my handwritten texts - King James Genesis 29 is appearantely fully AI written - a wall of text copied straight from chatgpt? Only partially AI written. - second chapter of Harry Potter? AI written parts In fact I could not find a sample yet which was not AI written at least partially according to th…
[deleted]
Re: Detect ChatGPT Generated Content
#68-- I copy and pasted GPT3 reply to - what are Jim Crow Laws? - said written by AI - then copy paste wikipeda first paragraph - said written by AI - then I wrote it myself - said written by AI - then I copy/paste 1 word over and over and over 100 or so times - it says "likely written by human but some by AI" then - highlights the whole text and notes - likely written by AI - hahhaa --
AI is so good that even human written text is indistinguishable from AI.
Re: Detect ChatGPT Generated Content
#69This is useless. I have tried it with 4 different samples and it made mistakes with each. - it thinks AI wrote parts of my handwritten texts - King James Genesis 29 is appearantely fully AI written - a wall of text copied straight from chatgpt? Only partially AI written. - second chapter of Harry Potter? AI written parts In fact I could not find a sample yet which was not AI written at least partially according to th…
Agreed. This is useless. I asked GPT-3 to write a HN post decrying GPTZero:
> GPTZero is absolute trash! It can't even detect plagarism correctly. It's a complete joke and waste of time! Don't bother with it! You'd be better off just manually checking for plagiarism. It's so unreliable it's not even worth mentioning. Just save yourself the time and money and don't bother with GPTZero. Worst AI tool ever!
GPTZero classified it as fully human written. This is a cat and mouse game where the mouse is always going to lose.
Re: Detect ChatGPT Generated Content
#70It would seem to be checking sentence structure, and if "too complex" -- embedded clauses and such -- it follows a hard-wired rule to report AI.