Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI shuts down its AI Classifier due to poor accuracy

decrypt.co

Post reply on HN
Print view View original

181–190 of 292 posts

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#181

Post by greiskul »

concurrentsquar wrote: post #171
bko wrote: post #157

Earlier quoted context omitted.

Copying a comment I posted a while ago: I listened to a podcast with Scott Aaronson that I'd highly recommend [0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together sl…

This, or cryptographic signing (like what the C2PA suggests) of all real digital media on the Earth are the only ways to maintain consensus reality ( https://en.wikipedia.org/wiki/Consensus_reality ) in a post-AI world. I personally would want to live in Aaronson's world, and not the world where a centralized authority controls the definition of reality.

How can we maintain consensus reality, when it has never existed? There are a couple of bubbles of humanity where honesty and skepticism and valued. Everywhere else, at all moments of history, truth has been manipulated to subjugate people. Be it newspaper owned by polical families, priests, etc.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#182

Post by RandomLensman »

aleph_minus_one wrote: post #179
RandomLensman wrote: post #169

Earlier quoted context omitted.

Why even care if it is written by a machine or not? I am not sure it matters as much as people think.

> Why even care if it is written by a machine or not? I am not sure it matters as much as people think. You don't see the writing on the wall? OK, here is a big hint: it might make a huge difference from a legal perspective whether some "photo" showing child sexual abuse (CSA) was generated using a camera and a real, physical child, or by some AI image generator.

I don't think all jurisdictions make that distinction to start with and even if they did and societies really wanted to go there: not sure why a licensing regime on generators with associated cryptographic information in the images could not work. We don't have to be broadly permissive, if at all.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#183

Post by Qqqwxs »

chakintosh wrote: post #21

This tool's been fueling tons of false accusations in academia. Wife is doing her PhD and she often tells me stories about professors falsely accusing students of using ChatGPT.

A group of students at my university were claiming their papers were being marked by a LLM. They cited a classifier like the OP which they used on their feedback comments.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#184

Post by frumper »

dontreact wrote: post #176
frumper wrote: post #163

Earlier quoted context omitted.

It's a strawman because they aren't comparable to AI detection tests. A screening coming back as possible cancer will lead to follow up tests to confirm, or rule out. An AI detection test coming back as positive can't be refuted or further tested with any level of accuracy. It's a completely unverifiable test with a low accuracy.

You are moving the goalposts here. The original claim I am responding to is "A tool that gives incorrect and inconsistent results shouldn’t have any part of a decision making process." I agree that there are places where we shouldn't put AI and that checking whether something is an LLM or not is one of them. However I think the sentence above takes it way too far and breast cancer screening is a pretty clear example…

The thread is about tools to evaluate LLMs. Please re-read my comment in that light and generously assume I'm talking about that.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#186

Post by [deleted] »

esclerofilo wrote: post #150
malfist wrote: post #136

Earlier quoted context omitted.

That's not how cryptographic signing works. Cryptographic signing means "I wrote this" or "I created this". Sure you could sign an AI generated image as yourself. But you could not sign an image as being created by Getty or NYT

I believe that's not what they're saying. It's signing hardware, like a camera that signs every picture you take, so not even you can tamper with it without invalidating that signature. Naively, then, a signed picture would be proof that it was a real picture taken of a real thing. What GP is saying is that people would inevitably get the keys from the cameras, and then the whole thing would be pointless.

[deleted]

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#187

Post by feoren »

capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

Indeed it's not possible. Say you had a classifier that detected whether a given text was AI generated or not. You can easily plug this classifier into the end of a generative network trying to fool it, and even backpropagate all the way from the yes/no output to the input layer of the generative network. Now you can easily generate text that fools that classifier.

So such a model is doomed from the start, unless its parameters are a closely-guarded secret (and never leaked). Then it means it's foolable by those with access and nobody else. Which means there's a huge incentive for adversaries to make their own, etc. etc. until it's just a big arms race.

It's clear the actual answer needs to be: we need better automated tools to detect quality content, whatever that might mean, whether written by a human or an AI. That would be a godsend. And if it turned into an arms race, the arms we're racing each other to build are just higher-quality content.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#188

Post by antod »

mplewis wrote: post #168
ribosometronome wrote: post #153

Earlier quoted context omitted.

Retyping the essay from the chatgpt while actively rewording the occassional sentence seems like it would do it.

It's a bit suspicious to type an essay linearly from start to finish, though.

A bit like how us old timers had to write our exams in the pen and paper days.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#189

Post by JohnFen »

RandomLensman wrote: post #169
constantcrying wrote: post #64

Earlier quoted context omitted.

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

Why even care if it is written by a machine or not? I am not sure it matters as much as people think.

There are a number of reasons people may care. For instance, the thing about art that appeals to me is that it's human communication. If it's machine generated, then I want to know so that I can properly contextualize it (and be able to know whether or not I'm supporting a real person by paying for it).

A world where I can't tell if something is made by human or by machine is a world that has been drained of something important to me. It would reduce the appeal of all art for me and render the world a bit less meaningful.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#190

Post by Nickersf »

wmeredith wrote: post #164
ribosometronome wrote: post #153

Earlier quoted context omitted.

Retyping the essay from the chatgpt while actively rewording the occassional sentence seems like it would do it.

It seems like that's nearing the sweet spot of fraud prevention, where committing the act of fraud is as much work as doing the real thing.

The intellectual labor of thinking about an essay, drafting it, editing, and revising it is much higher than strategically re-typing a ChatGPT output. One requires experience, knowledge, understanding and creativity, the other one requires basic functioning of motor skills and senses.

You could program a robot to re-type the ChatGPT output into a different word processor and feed it parameters to make the duration between keystrokes and backspaces fluctuate over time. You could even have it stop, come back later, copy and paste sections and re-organize as it moves through and end up with the final essay from ChatGPT.

Post reply on HN