Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI shuts down its AI Classifier due to poor accuracy

decrypt.co

Post reply on HN
Print view View original

151–160 of 292 posts

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#151

Post by jedberg »

Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it.

Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits".

A clever and unethical student is pretty much undetectable no mater what roadblocks you put in their way. This just stops the not clever ones. :)

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#152

Post by justrealist »

jedberg wrote: post #151

Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it. Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits". A clever and unethical student is pretty muc…

Well, there is one way, which is timed, proctored exams.

Which sucks, because take-home projects are evaluating a different skill set, and some people thrive on one vs the other. But it is what it is.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#153

Post by ribosometronome »

jedberg wrote: post #151

Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it. Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits". A clever and unethical student is pretty muc…

Retyping the essay from the chatgpt while actively rewording the occassional sentence seems like it would do it.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#155

Post by cwkoss »

esclerofilo wrote: post #150
malfist wrote: post #136

Earlier quoted context omitted.

That's not how cryptographic signing works. Cryptographic signing means "I wrote this" or "I created this". Sure you could sign an AI generated image as yourself. But you could not sign an image as being created by Getty or NYT

I believe that's not what they're saying. It's signing hardware, like a camera that signs every picture you take, so not even you can tamper with it without invalidating that signature. Naively, then, a signed picture would be proof that it was a real picture taken of a real thing. What GP is saying is that people would inevitably get the keys from the cameras, and then the whole thing would be pointless.

Yep.

A chain of trust is one way to solve this problem. Chains of trust aren't perfect, but they can work.

But if you're going to build a chain of trust that relies on humans to certify they used a non-tampered-with crypto camera, why not just let them use plain ol cameras. Adding cryptosigning hardware just adds a false sense of security that grifter salespeople will lie and say is 'impossible to break', and non-technical decision makers wont understand the threat model.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#156

Post by dontreact »

a13o wrote: post #55
dontreact wrote: post #43

Earlier quoted context omitted.

If you were trying to predict the direction a stock will move (up or down) and it was right 99.9% of the time, would you use it or not?

This is a strawman. First, the AI detection algorithms can't offer anything close to 99.9%. Second, your scenario doesn't analyze another human and issue judgement, as the AI detection algorithms do. When a human is miscategorized as a bot, they could find themselves in front of academic fraud boards, skipped over by recruiters, placed in the spam folder, etc.

It's not a strawman. There are many fundamentally unpredictable things where we can't make the benchmark be 100% accuracy.

To make it more concrete on work I am very familiar with: breast cancer screening. If you had a model that outperformed human radiologists at predicting whether there is pathology confirmed cancer within 1 year, but the accuracy was not 100%, would you want to use that model or not?

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#157

Post by bko »

constantcrying wrote: post #64
capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

Copying a comment I posted a while ago:

I listened to a podcast with Scott Aaronson that I'd highly recommend [0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together slightly more than random and you can mathematically derive with some level of certainty whether an output or even a section of an output was generated by the LLM. It's really clever and apparently he has a working prototype in development.

Some work arounds he hasn't figured out yet is asking for an output in language X and then translating it into language Y. But those may still be eventually figured out.

I think watermarking would be a big step forward to practical AI safety and ideally this method would be adopted by all major LLMs.

That part starts around 1 hour 25 min in.

> Scott Aaronson: Exactly. In fact, we have a pseudorandom function that maps the N-gram to, let’s say, a real number from zero to one. Let’s say we call that real number ri for each possible choice i of the next token. And then let’s say that GPT has told us that the ith token should be chosen with probability pi.

https://axrp.net/episode/2023/04/11/episode-20-reform-ai-ali...

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#158

Post by marcell »

jedberg wrote: post #151

Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it. Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits". A clever and unethical student is pretty muc…

This will be hard to break. It’s basically an hour long CAPTCHA. You can look at things like key stroke timing, mouse movement, revision pattern, etc. I don’t see LLM’s breaking this approach to classify human writing.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#160

Post by paxys »

chakintosh wrote: post #21

This tool's been fueling tons of false accusations in academia. Wife is doing her PhD and she often tells me stories about professors falsely accusing students of using ChatGPT.

It isn't just this one. There are a hundred different "AI detectors" sold online that are all basically snake oil, but overzealous professors and school administrators will keep paying for them regardless.
Post reply on HN