This tool's been fueling tons of false accusations in academia. Wife is doing her PhD and she often tells me stories about professors falsely accusing students of using ChatGPT.
OpenAI shuts down its AI Classifier due to poor accuracy
21–30 of 292 posts
Re: OpenAI shuts down its AI Classifier due to poor accuracy
#22Earlier quoted context omitted.
How would you go about watermarking AI written text?
One way is trying to sneak in a specific structure/pattern that is difficult for a human to notice when reading, like using a particular sentence length, paragraph length, or punctuation pattern. Or use certain words in the text that may not be frequently used by humans etc. Watermarking needs to be subtle enough to be unnoticeable to opposing parties, yet distinctive enough to be detectable. So, this is an arms race…
> One way is trying to sneak in a specific structure/pattern that is difficult for a human to notice when reading
This seems like a total non-starter. That can only negatively impact the answers. A solution needs to be totally decoupled from answer quality.
Re: OpenAI shuts down its AI Classifier due to poor accuracy
#23It's an open ended red-queen problem. You can't win.
Besides, even if they did win, they would still lose by shooting their own foot.
Re: OpenAI shuts down its AI Classifier due to poor accuracy
#24Watermarking is a more tractable approach, but the cat is out of the bag.
How would you go about watermarking AI written text?
Make half of the tokens (the AI's "dictionary") slightly more likely.
This would not impact output quality much, but it would only work for longish outputs. And the token probability "key" could probsbly be reverse engineered with enough output.
Re: OpenAI shuts down its AI Classifier due to poor accuracy
#25I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…
There's also the post going around about how it can (and does) falsely flag human posts as AI output, particularly among some autistic people. About as useful as a polygraph, no?
We could combine those, couldn't we?
Re: OpenAI shuts down its AI Classifier due to poor accuracy
#26Earlier quoted context omitted.
Invisible characters in a specific bit-pattern. Pretty common steganographic technique, really.
Can you elaborate on "invisible?" The only invisible character I can imagine is a space. It seems like any other character either isn't invisible or doesn't exist (ie, isnt a character). Additionally, if I copy-paste text like this are the invisible characters preserved? Are there a bunch of extra spaces somewhere?
There's a bunch of different "spaces", one is a "zero-width space" which isn't visible but still gets copied with the text.
Re: OpenAI shuts down its AI Classifier due to poor accuracy
#27Interesting. I was under the impression this tool was effective because of some sort of hidden patterns generated in sentences. I guess my assumption was way complex than what it actually is
Re: OpenAI shuts down its AI Classifier due to poor accuracy
#28Earlier quoted context omitted.
One way is trying to sneak in a specific structure/pattern that is difficult for a human to notice when reading, like using a particular sentence length, paragraph length, or punctuation pattern. Or use certain words in the text that may not be frequently used by humans etc. Watermarking needs to be subtle enough to be unnoticeable to opposing parties, yet distinctive enough to be detectable. So, this is an arms race…
> One way is trying to sneak in a specific structure/pattern that is difficult for a human to notice when reading This seems like a total non-starter. That can only negatively impact the answers. A solution needs to be totally decoupled from answer quality.
You can do this by injecting non visible unicode (LTR / RTL markers, zero width separators, the various "space" analogs, homographs of "normal" characters) but it can obviously be stripped out.
Re: OpenAI shuts down its AI Classifier due to poor accuracy
#29I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…
Taking away tools don't seem to me like the best response same way taking away things tends never to be. If the problem is people not using it right, that seems to me like it would be designed wrong for what people need it for. Like if the issue is using it wrong with too little sentences, then put a minimum sentence or something to have that minimum likelihood.
Same goes for representing what it means. If people don't understand statistics or math and such, then show what it means with circles or coins or stuff like that. Point is don't seem ever a good thing for options to get removed, especially if it's for bein cynical and judgin people like they're beneath deservin it. Don't make no sense.
Re: OpenAI shuts down its AI Classifier due to poor accuracy
#30Earlier quoted context omitted.
There's also the post going around about how it can (and does) falsely flag human posts as AI output, particularly among some autistic people. About as useful as a polygraph, no?
We could combine those, couldn't we?
You could but is there any reason to believe these two noisy signals wouldn't result in more combined noise than signal?
Sure, it's theoretically possible to add two noisy signals that are uncorrelated and get noise reduction, but is it probable this would be such a case?