Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI shuts down its AI Classifier due to poor accuracy

decrypt.co

Post reply on HN
Print view View original

231–240 of 292 posts

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#231

Post by deadbeeves »

jdshaffer wrote: post #217
RandomLensman wrote: post #169

Earlier quoted context omitted.

Why even care if it is written by a machine or not? I am not sure it matters as much as people think.

Well, I teach English as a second language in a non-English speaking country. I often used short essays and diary-writing for homework. The students have had lots of English input over the years, but not much experience with output. So, writing assignments work out very well for them. Alas, with ChatGPT on the rise here, they no longer have to write it themselves. The upshot of which is, the useful writing assignment…

Is your objective that your students learn, or to police them? In the latter case, yes, you have those two options you mentioned. In the former case, you can just continue as you were. Some will cheat and some will not. The ones who do are only cheating themselves.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#232

Post by burtonator »

capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

It's really disturbing to me how many people don't realize it's not possible.

It's like asking a 747 to be made into a dog.

It's completely nonsensical to me.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#233

Post by burtonator »

jedberg wrote: post #151

Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it. Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits". A clever and unethical student is pretty muc…

It's like hearing stories about people hating the first automobiles because they weren't horses.

The education bubble is about to implode - it will probably be one of the first industries killed by AI.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#234

Post by nradov »

capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

Dr. Michio Kaku claimed in an interview that it may eventually be possible for quantum computers to guarantee a certain level of truthfulness. I didn't really follow his argument and it seemed a little hand-wavey, but I can't prove that he's wrong.

https://mkaku.org/home/tag/quantum-computing/

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#235

Post by sobellian »

feoren wrote: post #187
capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

Indeed it's not possible. Say you had a classifier that detected whether a given text was AI generated or not. You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier. So such a model is doomed from the start, unless its…

Even without a GAN, it's quite possible that one could simply write `while(rejects(output)) { output = gpt(prompt) }` and obtain a sufficient output after a reasonable number of iterations.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#236

Post by nancyhn »

capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

I agree, transparency is essential, especially when it comes to AI applications. Many underestimate the complexity of distinguishing AI-written content from human-written, especially for short texts. There's a danger in trusting tools claiming to provide absolute certainty in this regard; no current technology can guarantee 100% accuracy. This incident underscores the need for a more realistic understanding of AI capabilities and limitations in text generation detection.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#237

Post by [deleted] »

constantcrying wrote: post #64
capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

[deleted]

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#238

Post by boondoggle16 »

klabb3 wrote: post #89
hackernewds wrote: post #15

Good. If it is not reliable, it's a further harm than good if it exists as a false security. An analogous example: my local pizza delivery (where I worked) would shut the box with a safety sticker, to avoid tampering / dipping by the delivery boys. Now, sometimes they would forget to do this for various logistical reasons. Every one of the non-stickered ones started getting returned as customers worried a pepperoni s…

It’s a law of nature that pepperoni thieves cannot take a job at a pizza place. They are forever doomed to be delivery guys.

This is actually kinda true with doordash etc. Those drivers are completely unvetted, they don't even have an interview.

The kind of people that can't get a job at a pizza place.

Personally, I never order delivery through these services. The incentives are all wrong. Not to mention the costs are super high: restaurants don't make any money, I pay out the @$$, and the drivers are given sub-minimum-wage pay after taking on the risks of delivery driving.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#239

Post by chinchilla2020 »

asddubs wrote: post #209
feoren wrote: post #187

Earlier quoted context omitted.

Indeed it's not possible. Say you had a classifier that detected whether a given text was AI generated or not. You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier. So such a model is doomed from the start, unless its…

The whole problem with AI is that it's able to copy some of the superficial indicators of quality content while feeding you lies. You cannot detect quality content without detecting truthfulness. Any heuristic you use in place of that can be copied without actually providing value (which is exactly what ChatGPT does now, when it gets things wrong)

That's the whole problem with LLMs in general. They are designed to be convincing, not necessarily accurate.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#240

Post by albert_e »

Many comments here seem to suggest that practically it is going to be impossible to classify text as Human generated versus AI generated -- given the many different ways in which such attempts can be foiled in a never ending game of cat-and-mouse.

If we accept this ...

The challenge I am foreseeing is this:

We are only at the very beginning of the AI revolution -- and if LLMs need to get more sophisticated and powerful in future they will need good-quality human-generated / curated training data at a scale that is likely impossible to do manual curation/cleansing/quality-checks on.

And there is no doubt that evey medium is going to get bombarded and spammed with AI-generated content in coming years.

How then, are we going to filter the data -- to separate the real data from AI generated noise -- to train future LLMs on -- and really push them to their potential.

This problem has been bugging me for a while and I commented here previously as well, tentatively calling it 'Data Pollution' for the lack of a better word.

Curious to hear other perspectives on this.

Post reply on HN