Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI shuts down its AI Classifier due to poor accuracy

decrypt.co

Post reply on HN
Print view View original

201–210 of 292 posts

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#201

Post by andromaton »

jedberg wrote: post #151

Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it. Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits". A clever and unethical student is pretty muc…

Proof of work, human version.

Yes, anybody can write an agent to meander about typing the chatgpt generated text into Google docs. Yes, Google could judge how likely it's that a document was typed by a human, but they won't for the same reasons openAI just cancelled this.

Somebody (maybe reacting to this news, maybe reading this thread) will write such an editor or evaluator. Another solution is screen recording as you write. Another (the best one, and the hardest one for educators) is to not request or grade things a robot can write better than most humans.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#202

Post by jwie »

jedberg wrote: post #151

Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it. Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits". A clever and unethical student is pretty muc…

Easy thing to do would be very low maximum word counts. ChatGPT is incapable of brevity.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#205

Post by tessierashpool »

feoren wrote: post #187
capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

Indeed it's not possible. Say you had a classifier that detected whether a given text was AI generated or not. You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier. So such a model is doomed from the start, unless its…

> You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier

could you contextualize your use of the word "easily" here?

I feel like "easily" might mean "with infinite funds and frictionless spherical developers."

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#206

Post by alexeldeib »

tessierashpool wrote: post #205
feoren wrote: post #187

Earlier quoted context omitted.

Indeed it's not possible. Say you had a classifier that detected whether a given text was AI generated or not. You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier. So such a model is doomed from the start, unless its…

> You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier could you contextualize your use of the word "easily" here? I feel like "easily" might mean "with infinite funds and frictionless spherical developers."

GANs are established engineering. Infinite funds and frictionless spheres aside, you don't need to break ground, but copy/paste/glue existing code with some comprehension.

LLMs are newer than GANs afaik, it just so happens GANs are a good fit here, not that one is "smarter" or "dumber".

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#208

Post by claudiawerner »

aleph_minus_one wrote: post #179
RandomLensman wrote: post #169

Earlier quoted context omitted.

Why even care if it is written by a machine or not? I am not sure it matters as much as people think.

> Why even care if it is written by a machine or not? I am not sure it matters as much as people think. You don't see the writing on the wall? OK, here is a big hint: it might make a huge difference from a legal perspective whether some "photo" showing child sexual abuse (CSA) was generated using a camera and a real, physical child, or by some AI image generator.

I agree with you, but in some jurisdictions the distance between stuff generated with AI and actual photographs of child abuse are treated rather closely; either way, possessing either could result in what the England & Wales calls a "sexual harm prevention order" (SHPO). To me the idea that someone could be served such an order without ever possessing real CSEM (or "child porn"), never mind actually never being near a child is rather worrying.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#209

Post by asddubs »

feoren wrote: post #187
capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

Indeed it's not possible. Say you had a classifier that detected whether a given text was AI generated or not. You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier. So such a model is doomed from the start, unless its…

The whole problem with AI is that it's able to copy some of the superficial indicators of quality content while feeding you lies. You cannot detect quality content without detecting truthfulness. Any heuristic you use in place of that can be copied without actually providing value (which is exactly what ChatGPT does now, when it gets things wrong)

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#210

Post by bloppe »

Rather than try to detect if pieces were generated by AI, why not just check if they're plagiarized off the outputs of a bunch of popular models? We already have traditional plagiarism analysis methods, we just need a corpus of most of the recent outputs from the most popular LLM services to check against. OpenAI, Google, Anthropic, and any other LLM-as-a-service companies could profitably sell access to these corpora to third-party analysis services that compile them and offer search.
Post reply on HN