Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI shuts down its AI Classifier due to poor accuracy

decrypt.co

Post reply on HN
Print view View original

211–220 of 292 posts

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#211

Post by TulliusCicero »

marcell wrote: post #158
jedberg wrote: post #151

Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it. Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits". A clever and unethical student is pretty muc…

This will be hard to break. It’s basically an hour long CAPTCHA. You can look at things like key stroke timing, mouse movement, revision pattern, etc. I don’t see LLM’s breaking this approach to classify human writing.

Seems easy to me? Just manually copy the text by typing it in yourself. There may be certain patterns that could potentially give it away vs truly human-generated text, but will Google Docs revision history show that level of detail?

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#212

Post by TulliusCicero »

wmeredith wrote: post #164
ribosometronome wrote: post #153

Earlier quoted context omitted.

Retyping the essay from the chatgpt while actively rewording the occassional sentence seems like it would do it.

It seems like that's nearing the sweet spot of fraud prevention, where committing the act of fraud is as much work as doing the real thing.

Doesn't sound like it to me. Researching a topic can be a lot of effort, while typing is easy, even with occasional rephrasing. Hell, even with constant rephrasing it's not very hard.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#213

Post by TulliusCicero »

mplewis wrote: post #168
ribosometronome wrote: post #153

Earlier quoted context omitted.

Retyping the essay from the chatgpt while actively rewording the occassional sentence seems like it would do it.

It's a bit suspicious to type an essay linearly from start to finish, though.

Will Google Docs provide that level of detail for revision history in a way that teachers can easily see that's it's likely AI-generated, and have high confidence in that?

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#214

Post by teaearlgraycold »

constantcrying wrote: post #170
bko wrote: post #157

Earlier quoted context omitted.

Copying a comment I posted a while ago: I listened to a podcast with Scott Aaronson that I'd highly recommend [0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together sl…

I think the chance of this working reliably is precisely zero. There are multiple trivial attacks against this and it can not work if the user has any kind of access to token level data (where he could trivially write his own truly random choice). And if there is a non-water marking neural network with enough capacity to do simple rewriting you can easily remove any watermark or the user does the minor rewrite himsel…

It’ll be the equivalent to a shutterstock watermark.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#215

Post by refulgentis »

jacquesm wrote: post #196
marcell wrote: post #158

Earlier quoted context omitted.

This will be hard to break. It’s basically an hour long CAPTCHA. You can look at things like key stroke timing, mouse movement, revision pattern, etc. I don’t see LLM’s breaking this approach to classify human writing.

> I don’t see LLM’s breaking this approach to classify human writing. Why not? Record a bunch of humans writing, train model, release. That's orders of magnitude simpler than to come up with the right text to begin with.

Lol. I love HN -- the reaction is because this is either straight-faced or tongue-in-cheek, if it's straight-faced, this is stylistically a parody of the infamous "well Dropbox is rsync, it's moat is basically a SWE-weekend" comment

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#216

Post by WiSaGaN »

constantcrying wrote: post #64
capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

The default style people cites about ChatGPT is also nothing intrinsic about AI, it is just this paticular AI is trained and prompted to output information in this way. The output style can change drastically with just a little prompt change even on the user side.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#217

Post by jdshaffer »

RandomLensman wrote: post #169
constantcrying wrote: post #64

Earlier quoted context omitted.

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

Why even care if it is written by a machine or not? I am not sure it matters as much as people think.

Well, I teach English as a second language in a non-English speaking country. I often used short essays and diary-writing for homework. The students have had lots of English input over the years, but not much experience with output. So, writing assignments work out very well for them. Alas, with ChatGPT on the rise here, they no longer have to write it themselves.

The upshot of which is, the useful writing assignments I used to give as homework will either have to be done in class (wasting valuable class time) or given up altogether (wasting valuable learning experiences).

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#218

Post by ShamelessC »

bloppe wrote: post #210

Rather than try to detect if pieces were generated by AI, why not just check if they're plagiarized off the outputs of a bunch of popular models? We already have traditional plagiarism analysis methods, we just need a corpus of most of the recent outputs from the most popular LLM services to check against. OpenAI, Google, Anthropic, and any other LLM-as-a-service companies could profitably sell access to these corpor…

One reason would be that LLM's don't tend to output verbatim pieces from their dataset (unless explicitly prompted to do so). This is made further complicated by the "temperature" setting allowing users to make output even more "creative" than its dataset.

In OpenAI's case, its writing style usually comes from OpenAI's in-house dataset they used for RLHF. This is what gives it the ability to chat and respond with its signature (perhaps overly formal and apologetic) tone.

Although it can be used to write in other styles, sometimes it will refuse to because of this.

Post reply on HN