Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI shuts down its AI Classifier due to poor accuracy

decrypt.co

Post reply on HN
Print view View original

51–60 of 292 posts

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#51

Post by 13years »

"Half a year later, that tool is dead, killed because it couldn’t do what it was designed to do."

This was my conclusion as well testing the image detectors.

Current automated detection isn’t very reliable. I tried out Optic’s AI or Not , which boasts 95% accuracy, on a small sample of my own images. It correctly labeled those with AI content as AI generated, but it also labeled about 50% of my own stock photo composites I tried as AI generated. If generative AI was not a moving target I would be optimistic such tools could advance and become highly reliable. However, that is not the case and I have doubts this will ever be a reliable solution.

from my article on AI art - https://www.mindprison.cc/p/ai-art-challenges-meaning-in-a-w...

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#52

Post by cjbgkagh »

BestGuess wrote: post #44
cjbgkagh wrote: post #41

Earlier quoted context omitted.

What about the patients getting unnecessary treatments? How upset should they be? What about the student expelled for AI plagiarism due to a false reading? These things are unreliable, and despite an infinite amount of caveats there is no way to prevent people from over relying on it. We might as well dunk people in the water to see if they float. That’s a weird kind of extortion, a demand that we placate a subset of…

I don't see how that's any different from anything, any tool, any power, any method. Same problem with everything. That's why this don't convince me and just seems like removing things cynically instead of improving it. Seems to me like the company also really don't want its service identified negatively like that and get itself associated with cheaters even if they're the ones selling the cheat identifying, or somet…

Firstly, this tool cannot be made better than it is due to the nature of its construction, it is completely intrinsic. Secondly, as LLM models improve, as they are guaranteed to do, this tool can only become worse as it becomes increasingly difficult to distinguish between human and AI written text.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#53

Post by RandomLensman »

BestGuess wrote: post #29
capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

Taking away tools don't seem to me like the best response same way taking away things tends never to be. If the problem is people not using it right, that seems to me like it would be designed wrong for what people need it for. Like if the issue is using it wrong with too little sentences, then put a minimum sentence or something to have that minimum likelihood. Same goes for representing what it means. If people don…

I think it will be increasingly irrelevant what specific process generated a text, for example. Already before genAI people did not in general query into how politicians' speeches were crafted etc.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#54

Post by kylecordes »

There is inherent conflict in having both an AI tool business and an AI tool detection business.

If the first does a good job, the second fails. And vice versa.

(On the other hand, maybe there is a lot of money to be made selling both, to different groups?)

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#55

Post by a13o »

dontreact wrote: post #43
frumper wrote: post #36

Earlier quoted context omitted.

A tool that gives incorrect and inconsistent results shouldn’t have any part of a decision making process. There is no way to know when it’s wrong so you’ll either use it to help justify what you want, or ignore it. Edit: this tool is as reliable as a magic 8-ball

If you were trying to predict the direction a stock will move (up or down) and it was right 99.9% of the time, would you use it or not?

This is a strawman. First, the AI detection algorithms can't offer anything close to 99.9%. Second, your scenario doesn't analyze another human and issue judgement, as the AI detection algorithms do.

When a human is miscategorized as a bot, they could find themselves in front of academic fraud boards, skipped over by recruiters, placed in the spam folder, etc.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#56

Post by hn_go_brrrrr »

frumper wrote: post #36
capableweb wrote: post #11

Earlier quoted context omitted.

Both false-positives are as useful as the other one, flagged "human" but actually "LLM" vs flagged "LLM" but actually "human". As long as no one put too much weight on the result, no harm would have been done, in either case. But clearly, people can't stay away from jumping to conclusions based on what a simple-but-incorrect tool says.

A tool that gives incorrect and inconsistent results shouldn’t have any part of a decision making process. There is no way to know when it’s wrong so you’ll either use it to help justify what you want, or ignore it. Edit: this tool is as reliable as a magic 8-ball

This is an unreasonable standard. Outside of trivial situations, there are no infallible tools.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#57

Post by frumper »

bgirard wrote: post #49
frumper wrote: post #36

Earlier quoted context omitted.

A tool that gives incorrect and inconsistent results shouldn’t have any part of a decision making process. There is no way to know when it’s wrong so you’ll either use it to help justify what you want, or ignore it. Edit: this tool is as reliable as a magic 8-ball

> A tool that gives incorrect and inconsistent results shouldn’t have any part of a decision making process. It can be used for some decision (i.e. not critical ones), but it should NOT be used to accused someone of academic misconduct unless the tool meets a very robust quality standard. > this tool is as reliable as a magic 8-ball Citation needed

The AI tool doesn't give accurate results. You don't know when it's not accurate. There is no accurate way to check its results. Who should use a tool to help them make a decision when you don't know when the tool will be wrong and it has a low rate of accuracy? It's in the article.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#58

Post by sakopov »

Humorously, in my experience, if a response from ChatGPT ever got classified as AI generated by tools like ZeroGPT or similar, all I had to do was adjust the prompt to tell the model not to sound like it was AI generated and that bypassed all detection with a very high success rate. Additionally, I also found that if you prompt it to make the response be in the style or some known writer for example, it often made responses 100% human written by most AI detection models.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#59

Post by BestGuess »

cjbgkagh wrote: post #52
BestGuess wrote: post #44

Earlier quoted context omitted.

I don't see how that's any different from anything, any tool, any power, any method. Same problem with everything. That's why this don't convince me and just seems like removing things cynically instead of improving it. Seems to me like the company also really don't want its service identified negatively like that and get itself associated with cheaters even if they're the ones selling the cheat identifying, or somet…

Firstly, this tool cannot be made better than it is due to the nature of its construction, it is completely intrinsic. Secondly, as LLM models improve, as they are guaranteed to do, this tool can only become worse as it becomes increasingly difficult to distinguish between human and AI written text.

I don't know about neither of those. How is it intrinsic? What stops detection improving just because AI gets better? Assuming it just doesn't become sentient human replica or something I mean AI like this where it's just a language model thing. Plus that's assuming future stuff you can track in the meanwhile and still don't justify "remove it because people dumb and do bad stuff with tool", that'd only justify removing it later as they do get better.
Post reply on HN