Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI shuts down its AI Classifier due to poor accuracy

decrypt.co

Post reply on HN
Print view View original

11–20 of 292 posts

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#11

Post by capableweb »

catboybotnet wrote: post #8
capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

There's also the post going around about how it can (and does) falsely flag human posts as AI output, particularly among some autistic people. About as useful as a polygraph, no?

Both false-positives are as useful as the other one, flagged "human" but actually "LLM" vs flagged "LLM" but actually "human". As long as no one put too much weight on the result, no harm would have been done, in either case. But clearly, people can't stay away from jumping to conclusions based on what a simple-but-incorrect tool says.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#12

Post by hospitalJail »

Not to say "I can detect chatgpt" but it sure seems to have a similar way of talking even when I say things like: Talk like a "Millennial male who is obsessed with Zelda, their name is bob zelenski"

Now the topic isnt about anything millennial or Zelda related, but I'd think that the language model would select sentence and paragraph phrasing differently.

Maybe I need to switch to the API.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#13

Post by thewataccount »

sydon wrote: post #3
seeknotfind wrote: post #2

Watermarking is a more tractable approach, but the cat is out of the bag.

How would you go about watermarking AI written text?

https://arxiv.org/pdf/2301.10226.pdf

Here's a decent paper on it.

It covers private watermarking (you can't detect it exists without a key), resistance to modifications, etc. Essentially you wouldn't know it was there and you can't make simple modifications to fool it.

OpenAI could already be doing this, and they could be watermarking with your account ID if they wanted to.

The current best countermeasure is likely paraphrasing attacks https://arxiv.org/pdf/2303.11156.pdf

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#14

Post by doctorpangloss »

sydon wrote: post #3
seeknotfind wrote: post #2

Watermarking is a more tractable approach, but the cat is out of the bag.

How would you go about watermarking AI written text?

I don't know.

I suppose hosted solutions like ChatGPT could offer an API where you copy some text in, and it searches its history of generated content to see if anything matches.

> bUt aCtuAlLy...

It's not like I don't know the bajillion limitations here. There are many audiences for detection. All of them are XY Problems. And the people asking for this stuff don't participate on Hacker News aka Unpopular Opinions Technology Edition.

There will probably be a lot of "services" that "just" "tell you" if "it" is "written by an AI."

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#15

Post by hackernewds »

Good. If it is not reliable, it's a further harm than good if it exists as a false security.

An analogous example: my local pizza delivery (where I worked) would shut the box with a safety sticker, to avoid tampering / dipping by the delivery boys. Now, sometimes they would forget to do this for various logistical reasons. Every one of the non-stickered ones started getting returned as customers worried a pepperoni stolen. They stopped doing it shortly after.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#17

Post by merlincorey »

sydon wrote: post #3
seeknotfind wrote: post #2

Watermarking is a more tractable approach, but the cat is out of the bag.

How would you go about watermarking AI written text?

Invisible characters in a specific bit-pattern.

Pretty common steganographic technique, really.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#18

Post by philipov »

merlincorey wrote: post #17
sydon wrote: post #3

Earlier quoted context omitted.

How would you go about watermarking AI written text?

Invisible characters in a specific bit-pattern. Pretty common steganographic technique, really.

So, all I have to do is copy-paste it into a text editor with remove-all-formatting to circumvent that?

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#19

Post by nonethewiser »

merlincorey wrote: post #17
sydon wrote: post #3

Earlier quoted context omitted.

How would you go about watermarking AI written text?

Invisible characters in a specific bit-pattern. Pretty common steganographic technique, really.

Can you elaborate on "invisible?" The only invisible character I can imagine is a space. It seems like any other character either isn't invisible or doesn't exist (ie, isnt a character).

Additionally, if I copy-paste text like this are the invisible characters preserved? Are there a bunch of extra spaces somewhere?

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#20

Post by cateye »

sydon wrote: post #3
seeknotfind wrote: post #2

Watermarking is a more tractable approach, but the cat is out of the bag.

How would you go about watermarking AI written text?

One way is trying to sneak in a specific structure/pattern that is difficult for a human to notice when reading, like using a particular sentence length, paragraph length, or punctuation pattern. Or use certain words in the text that may not be frequently used by humans etc.

Watermarking needs to be subtle enough to be unnoticeable to opposing parties, yet distinctive enough to be detectable.

So, this is an arms race especially because detecting it and altering it based on the watermark is also fun :)

Post reply on HN