Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI shuts down its AI Classifier due to poor accuracy

decrypt.co

Post reply on HN
Print view View original

171–180 of 292 posts

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#171

Post by concurrentsquar »

bko wrote: post #157
constantcrying wrote: post #64

Earlier quoted context omitted.

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

Copying a comment I posted a while ago: I listened to a podcast with Scott Aaronson that I'd highly recommend [0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together sl…

This, or cryptographic signing (like what the C2PA suggests) of all real digital media on the Earth are the only ways to maintain consensus reality (https://en.wikipedia.org/wiki/Consensus_reality) in a post-AI world.

I personally would want to live in Aaronson's world, and not the world where a centralized authority controls the definition of reality.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#172

Post by KeplerBoy »

gigglesupstairs wrote: post #128
VeninVidiaVicii wrote: post #99

Earlier quoted context omitted.

Eh I am doing my PhD and I use ChatGPT all the time!

And? The tool in question was used for AI text detection not generation.

Not all people might be accused wrongly. Then again does it matter if you use ChatGPT for inspiration?

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#173

Post by air7 »

bko wrote: post #157
constantcrying wrote: post #64

Earlier quoted context omitted.

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

Copying a comment I posted a while ago: I listened to a podcast with Scott Aaronson that I'd highly recommend [0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together sl…

I heard of this (very neat) idea and gave it some thought. I think it can work very well in the short term. Perhaps OpenAI has already implemented this and can secretly detect long enough text created by GPT with high levels of accuracy.

However, as soon a detection tool becomes publicly available (or even just the knowledge that watermarking has been implemented internally), a simple enough garbling LLM would pop up that would only need to be smart enough to change words and phrasing here and there.

Of course these garbling LLMs could have a watermark of their own... So it might turn out to be a kind of cat-and-mouse game but with strong bias towards the mouse, as FOSS versions of garblers would be created or people would actually do some work manually, and make the changes by hand.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#174

Post by ummonk »

bko wrote: post #157
constantcrying wrote: post #64

Earlier quoted context omitted.

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

Copying a comment I posted a while ago: I listened to a podcast with Scott Aaronson that I'd highly recommend [0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together sl…

This would be trivially broken once sufficiently good open source pretrained LLMs become available, as bad actors would simply use unwatermarked models.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#175

Post by ummonk »

sebzim4500 wrote: post #88
rootusrootus wrote: post #81

Good. And I think watermarking AI output is also a dead end. Better that we simply assume that all content is fake unless proven otherwise. To the extent that we need trustworthy photos, it seems like a better idea to cryptographically sign images at the hardware level when the photo is taken. Voluntarily watermarking AI content is completely pointless.

I can see that working for specialized equipment like police body cameras, but if every camera manufacturer in the world needs to manage keys and install them securely into their sensors then there will be leaked keys within weeks.

Just use a certificate chain. The manufacturer can provide each camera its own private key, signed by the manufacturer.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#176

Post by dontreact »

frumper wrote: post #163
dontreact wrote: post #156

Earlier quoted context omitted.

It's not a strawman. There are many fundamentally unpredictable things where we can't make the benchmark be 100% accuracy. To make it more concrete on work I am very familiar with: breast cancer screening. If you had a model that outperformed human radiologists at predicting whether there is pathology confirmed cancer within 1 year, but the accuracy was not 100%, would you want to use that model or not?

It's a strawman because they aren't comparable to AI detection tests. A screening coming back as possible cancer will lead to follow up tests to confirm, or rule out. An AI detection test coming back as positive can't be refuted or further tested with any level of accuracy. It's a completely unverifiable test with a low accuracy.

You are moving the goalposts here. The original claim I am responding to is "A tool that gives incorrect and inconsistent results shouldn’t have any part of a decision making process."

I agree that there are places where we shouldn't put AI and that checking whether something is an LLM or not is one of them. However I think the sentence above takes it way too far and breast cancer screening is a pretty clear example of somewhere we should accept AI even if it can sometimes make mistakes.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#178

Post by constantcrying »

air7 wrote: post #173
bko wrote: post #157

Earlier quoted context omitted.

Copying a comment I posted a while ago: I listened to a podcast with Scott Aaronson that I'd highly recommend [0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together sl…

I heard of this (very neat) idea and gave it some thought. I think it can work very well in the short term. Perhaps OpenAI has already implemented this and can secretly detect long enough text created by GPT with high levels of accuracy. However, as soon a detection tool becomes publicly available (or even just the knowledge that watermarking has been implemented internally), a simple enough garbling LLM would pop up…

There are already quite complex language models which can run on a CPU. Outside of the government banning personal LLMs, the chance of there not existing a working fully FOSS and open data rewrite model, if it becomes known that ChatGPT output is marked, seems very low.

The water marking techniques also can not work after some level of sophisticated rewriting. There simply will be no data encoded in the probabilities of the words.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#179

Post by aleph_minus_one »

RandomLensman wrote: post #169
constantcrying wrote: post #64

Earlier quoted context omitted.

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

Why even care if it is written by a machine or not? I am not sure it matters as much as people think.

> Why even care if it is written by a machine or not? I am not sure it matters as much as people think.

You don't see the writing on the wall? OK, here is a big hint: it might make a huge difference from a legal perspective whether some "photo" showing child sexual abuse (CSA) was generated using a camera and a real, physical child, or by some AI image generator.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#180

Post by matteoraso »

jedberg wrote: post #151

Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it. Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits". A clever and unethical student is pretty muc…

>Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits".

No need to complicate it that much. Just start off writing an essay normally, and then paste in the GPT output normally. A teacher probably isn't going to check any of the revision history, especially if there's more than 30 students to go through.

Post reply on HN