OpenAI shuts down its AI Classifier due to poor accuracy • Hacker News Board

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#141

Post by humanistbot » Tue, Jul 25, 2023, 7:09 PM UTC

spandrew wrote: post #140

The only way to prevent AI from answering questions in digital platforms is to develop a ML db on the typing style of every student across their tenure at an institution. Good luck getting that approved — departments can't even access grade or demo data without a steering group going through a 3-deep committee process. ¯\_(ツ)_/¯ try paper I guess. Time to brush up on our OCR.

If AI can replicate linguistic patterns in a way that is undetectable for both humans and models, then it seems even easier for a ML model to emulate a natural typing style, rhythm, and cadence in a way that is undetectable for both humans and models.

But you know who has more real-world data on typing style? Google, Microsoft, Meta, and everyone else who runs SaaS docs, emails, or messaging. I imagine a lot of students write their essays on Google Docs, Word, or the like, and submit them as attachments or copy-paste into a textbox.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#142

Post by RugnirViking » Tue, Jul 25, 2023, 7:09 PM UTC

BestGuess wrote: post #42

insanitybit wrote: post #38

Earlier quoted context omitted.

All that has to be shown is that the tool is as bad as or worse than random today , in order to remove it today.

From the article, "while incorrectly labeling the human-written text as AI-written 9% of the time." Seems like from what the article we're talkin about says it definitely ain't worse than random by far. Thing you most want to avoid is wrongly labeling humans as AI-written so that seems pretty good. Though it only identified 26% of AI text as "likely AI-written" that's still better than nothing, and better than random…

I'd want to see a lot better than "better than random" for the type of tool which is already being used to discipline students for academic misconduct, making hiring and firing decisions over who used AI in what CV/job tasks, and generally used to check if someone decieved others by passing off ai writing as their own, a wrong result can impugn people's reputations

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#143

Post by toss1 » Tue, Jul 25, 2023, 7:17 PM UTC

constantcrying wrote: post #64

capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

Yup.

Moreover, the way to deal with AI in this context is not like the way to deal with plagiarism; do not try to detect AI and punish its use.

Instead, assign it's use, and have the students critique the output and find the errors. This both builds skills in using a new technology, and more critically, builds the essential skills of vigilance for errors, and deeper understanding of the material — really helping students strengthen their BS detectors, a critical life skill.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#144

Post by andy99 » Tue, Jul 25, 2023, 7:25 PM UTC

capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

> The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high.

I saw that this report came out today which frankly is baffling: https://gpai.ai/projects/responsible-ai/social-media-governa... (Foundation AI Models Need Detection Mechanisms as a Condition of Release [pdf])

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#145

Post by whimsicalism » Tue, Jul 25, 2023, 7:27 PM UTC

SkyPuncher wrote: post #120

whimsicalism wrote: post #83

Earlier quoted context omitted.

Watermarking was never going to be successful except for the most naive uses.

It can likely work in images where you can make subtle, human-undetectable tweaks across thousands/millions of pixels, each with many possible values. Nearly impossible across data with a couple hundred characters and dozens to thousands of tokens.

right but the non-naive approach would be to add noise or have a dumber model rewrite the image. agreed it is easier with images though

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#146

Post by jillesvangurp » Tue, Jul 25, 2023, 7:30 PM UTC

This kind of thing strikes me in any case as something that's only good for the generation of AI it's been trained against. And with the exponential improvements happening almost on a monthly basis, that becomes obsolete pretty quickly and a bit of a moving target.

Maybe a better term would be Superior Intelligence (SI). I sure as hell would not be able to pass any legal or medical exams without dedicating the next decade or so to getting there. Nor do I have any interest in doing so. But chat gpt 4 is apparently able to wow its peers. Does that pass the Turing test because it's too smart or too stupid? Most of humanity would fail that test.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#147

Post by insanitybit » Tue, Jul 25, 2023, 7:32 PM UTC

JustBreath wrote: post #112

siglesias wrote: post #110

Earlier quoted context omitted.

I'd challenge this assumption. ChatGPT is supposed to convey information and answer questions in a manner that is intelligible to humans. It doesn't mean it should write indistinguishably from humans. It has a certain manner of prose that (to me) is distinctive and, for lack of a better descriptor, silkier, more anodyne, than most human writing. It should only attempt a distinct style if prompted to.

That's true, you could even purposely inject fingerprinting into its writing style and it could still accomplish the goal of conveying information to people.

All I would have to do is run the same tool over the text, see it gets flagged, and then modify the text until it no longer gets flagged. That's assuming I can't just prompt inject my way out of the scenario.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#148

Post by yk » Tue, Jul 25, 2023, 7:32 PM UTC

Funny incentive problem, OpenAI obviously has an incentive to use it's best AI detection tool for adversarial training, with the result that it's detection tool will not be very good against chatGPT generated text because it is trained to defeat the detection tool.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#149

Post by jstarfish » Tue, Jul 25, 2023, 7:35 PM UTC

michaelt wrote: post #33

nonethewiser wrote: post #19

Earlier quoted context omitted.

Can you elaborate on "invisible?" The only invisible character I can imagine is a space. It seems like any other character either isn't invisible or doesn't exist (ie, isnt a character). Additionally, if I copy-paste text like this are the invisible characters preserved? Are there a bunch of extra spaces somewhere?

When students try to evade plagiarism detectors, they will swap characters like replacing spaces with nonbreaking spaces, replacing letters with lookalikes (I vs extended Cyrillic Ӏ etc), and inserting things like the invisible 'Combining Grapheme Joiner' IMHO it isn't a feasible way of watermarking text though - as someone would promptly come up with a website that undid such substitutions.

> IMHO it isn't a feasible way of watermarking text though - as someone would promptly come up with a website that undid such substitutions.

It doesn't matter since there's no one-pass solution to counterfeiting.

You have the right of it-- the best you can hope for is adding more complexity to the product, which adds steps to their workflow and increases the chances of the counterfeiter overlooking any particular detail that you know to look for.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#150

Post by esclerofilo » Tue, Jul 25, 2023, 7:35 PM UTC

malfist wrote: post #136

cwkoss wrote: post #118

Earlier quoted context omitted.

Cryptography cant save us here, people will figure out how to send AI images to the crypto hardware to get it signed in months. Just would be another similar layer of false security.

That's not how cryptographic signing works. Cryptographic signing means "I wrote this" or "I created this". Sure you could sign an AI generated image as yourself. But you could not sign an image as being created by Getty or NYT

I believe that's not what they're saying. It's signing hardware, like a camera that signs every picture you take, so not even you can tamper with it without invalidating that signature. Naively, then, a signed picture would be proof that it was a real picture taken of a real thing. What GP is saying is that people would inevitably get the keys from the cameras, and then the whole thing would be pointless.