Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI shuts down its AI Classifier due to poor accuracy

decrypt.co

Post reply on HN
Print view View original

191–200 of 292 posts

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#191

Post by RandomLensman »

JohnFen wrote: post #189
RandomLensman wrote: post #169

Earlier quoted context omitted.

Why even care if it is written by a machine or not? I am not sure it matters as much as people think.

There are a number of reasons people may care. For instance, the thing about art that appeals to me is that it's human communication. If it's machine generated, then I want to know so that I can properly contextualize it (and be able to know whether or not I'm supporting a real person by paying for it). A world where I can't tell if something is made by human or by machine is a world that has been drained of somethin…

Fair, but I think that will shake out easier than expected: if there is a market (i.e. it is being valued) for certain things human generated people will work on being able to authenticate their output. Yes, there will likeky be fraud etc., but if there is a reasonable market it has a good chance of working because it serves all participants.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#192

Post by explaininjs »

ummonk wrote: post #175
sebzim4500 wrote: post #88

Earlier quoted context omitted.

I can see that working for specialized equipment like police body cameras, but if every camera manufacturer in the world needs to manage keys and install them securely into their sensors then there will be leaked keys within weeks.

Just use a certificate chain. The manufacturer can provide each camera its own private key, signed by the manufacturer.

And when the sensor bus is hijacked to directly feed “trusted” data into the processor?

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#193

Post by JohnFen »

mplewis wrote: post #168
ribosometronome wrote: post #153

Earlier quoted context omitted.

Retyping the essay from the chatgpt while actively rewording the occassional sentence seems like it would do it.

It's a bit suspicious to type an essay linearly from start to finish, though.

Is it? That's how I've always written them (and still do to this day). I write the first draft linearly from start to end, then go back and do my revising and editing.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#194

Post by explaininjs »

sebzim4500 wrote: post #96
zarzavat wrote: post #93

Earlier quoted context omitted.

There’s a much more effective way: store hashes of each output paragraph (with a minimum entropy) that has ever been generated, and allow people to enter a block of text to search the database. It wouldn’t beat determined users but it would at least catch the unaware.

Changing every 10th word defeats that strategy but doesn't defeat a cryptographic bias. Also the cost of storing every paragraph hash might eventually add up even if at the moment it would be negligable compared to the generation cost.

One solution is to store a hash of every n-gram for n from 2 to whatever, then report what percent of ngrams of various lengths were hits.

Did someone say Bloom Filter??

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#195

Post by motti »

constantcrying wrote: post #178
air7 wrote: post #173

Earlier quoted context omitted.

I heard of this (very neat) idea and gave it some thought. I think it can work very well in the short term. Perhaps OpenAI has already implemented this and can secretly detect long enough text created by GPT with high levels of accuracy. However, as soon a detection tool becomes publicly available (or even just the knowledge that watermarking has been implemented internally), a simple enough garbling LLM would pop up…

There are already quite complex language models which can run on a CPU. Outside of the government banning personal LLMs, the chance of there not existing a working fully FOSS and open data rewrite model, if it becomes known that ChatGPT output is marked, seems very low. The water marking techniques also can not work after some level of sophisticated rewriting. There simply will be no data encoded in the probabilities…

If it's sophisticatedly rewritten then it's no longer AI generated

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#196

Post by jacquesm »

marcell wrote: post #158
jedberg wrote: post #151

Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it. Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits". A clever and unethical student is pretty muc…

This will be hard to break. It’s basically an hour long CAPTCHA. You can look at things like key stroke timing, mouse movement, revision pattern, etc. I don’t see LLM’s breaking this approach to classify human writing.

> I don’t see LLM’s breaking this approach to classify human writing.

Why not? Record a bunch of humans writing, train model, release. That's orders of magnitude simpler than to come up with the right text to begin with.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#197

Post by BestGuess »

RugnirViking wrote: post #142
BestGuess wrote: post #42

Earlier quoted context omitted.

From the article, "while incorrectly labeling the human-written text as AI-written 9% of the time." Seems like from what the article we're talkin about says it definitely ain't worse than random by far. Thing you most want to avoid is wrongly labeling humans as AI-written so that seems pretty good. Though it only identified 26% of AI text as "likely AI-written" that's still better than nothing, and better than random…

I'd want to see a lot better than "better than random" for the type of tool which is already being used to discipline students for academic misconduct, making hiring and firing decisions over who used AI in what CV/job tasks, and generally used to check if someone decieved others by passing off ai writing as their own, a wrong result can impugn people's reputations

Wherever you draw the line someone's going to be upset at where the line is. You're echoing the other guy's concern, really everyone's concern. Same issue with everything from criminal justice to government all around so there's not really any value in yelling personal preferences at one another, even assumin I disagree which I don't. That ain't what I'm about in either case and it don't change what I said about removing options by assuming people suck being a bad way to go about doing anything.

Might as well remove all comment sections because people suck so assume there's no value having one. Pick any number of things like that. Just ain't a good way to go thinking about anything let alone defending a company for removing it, since the same logic justifies removing your ability to criticize or defend it in the first place. You an AI expert? Assume no, so why we let you talk about it? Or me? People suck so why let you comment? On and on like that.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#198

Post by vunderba »

bko wrote: post #157
constantcrying wrote: post #64

Earlier quoted context omitted.

Even the idea of it is bad, ChatGPT is supposed to write indistinguishably from a human. The "detector" has extremely little information and the only somewhat reasonable criteria are things like style, where ChatGPT certainly has a particular, but by no means unique writing style. And as it gets better it will (by definition) be better at writing in more varied styles.

Copying a comment I posted a while ago: I listened to a podcast with Scott Aaronson that I'd highly recommend [0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together sl…

Even if you could force the bad actors to use this watermarked large language model, there's no guarantee that they couldn't immediately feed that through Langchain into a different large language model that would render all the original watermarks useless.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#199

Post by woeirua »

lacker wrote: post #138

Smart of OpenAI to shut down a tool that basically doesn't work before the school year starts and students start to get in trouble based on it. I think this upcoming school year is going to be a wakeup call for many educators. ChatGPT with GPT-4 is already capable of getting mostly A's on Harvard essay assignments - the best analysis I have seen is this one: https://www.slowboring.com/p/chatgpt-goes-to-harvard I'm no…

Nah, everything is just going to be proctored exams on paper in the future. Sucks for the pro take-home project crowd, but they ruined it for themselves.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#200

Post by itsmoneyzzzzu »

capableweb wrote: post #5

I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could…

[dead]
Post reply on HN