Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI shuts down its AI Classifier due to poor accuracy

decrypt.co

Post reply on HN
Print view View original

91–100 of 292 posts

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#91

Post by frumper »

pixl97 wrote: post #85
hackernewds wrote: post #15

Good. If it is not reliable, it's a further harm than good if it exists as a false security. An analogous example: my local pizza delivery (where I worked) would shut the box with a safety sticker, to avoid tampering / dipping by the delivery boys. Now, sometimes they would forget to do this for various logistical reasons. Every one of the non-stickered ones started getting returned as customers worried a pepperoni s…

Eh, I'd consider that a failure of employee training and reverse the situation by giving out a weekly bonus to shifts that did not fail to put the security stickers on. Kinda like if they forgot to put the security seal on your aspirin, I'm not going to take them all off because someone forgot to run production with all the bottles sealed.

The bottle of Aspirin goes through many hands between the manufacturer and you including sitting on an unattended shelf open to the public. The person making the pizza is working for the same company as the person delivering it, or may even be the same person. If you can't trust the pizza co delivery person then you probably shouldn't trust the person making it either.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#93

Post by zarzavat »

sebzim4500 wrote: post #67
kylecordes wrote: post #54

There is inherent conflict in having both an AI tool business and an AI tool detection business. If the first does a good job, the second fails. And vice versa. (On the other hand, maybe there is a lot of money to be made selling both, to different groups?)

I don't think this follows. If they wanted, they could crypographically bias the sampling to make the output detectable without decreasing capabilities at all. Only people using it deceptively would be affected. No idea what portion of ChatGPT's users that is, would be very interested to know.

There’s a much more effective way: store hashes of each output paragraph (with a minimum entropy) that has ever been generated, and allow people to enter a block of text to search the database.

It wouldn’t beat determined users but it would at least catch the unaware.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#94

Post by klabb3 »

sakopov wrote: post #58

Humorously, in my experience, if a response from ChatGPT ever got classified as AI generated by tools like ZeroGPT or similar, all I had to do was adjust the prompt to tell the model not to sound like it was AI generated and that bypassed all detection with a very high success rate. Additionally, I also found that if you prompt it to make the response be in the style or some known writer for example, it often made re…

“Can’t you just try to blend in and be a little more cool? The bouncer is gonna notice.”

Starts talking like Shakespeare

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#95

Post by tudorw »

sebzim4500 wrote: post #88
rootusrootus wrote: post #81

Good. And I think watermarking AI output is also a dead end. Better that we simply assume that all content is fake unless proven otherwise. To the extent that we need trustworthy photos, it seems like a better idea to cryptographically sign images at the hardware level when the photo is taken. Voluntarily watermarking AI content is completely pointless.

I can see that working for specialized equipment like police body cameras, but if every camera manufacturer in the world needs to manage keys and install them securely into their sensors then there will be leaked keys within weeks.

Makes fake image, hold it in front of camera, click, verified image...

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#96

Post by sebzim4500 »

zarzavat wrote: post #93
sebzim4500 wrote: post #67

Earlier quoted context omitted.

I don't think this follows. If they wanted, they could crypographically bias the sampling to make the output detectable without decreasing capabilities at all. Only people using it deceptively would be affected. No idea what portion of ChatGPT's users that is, would be very interested to know.

There’s a much more effective way: store hashes of each output paragraph (with a minimum entropy) that has ever been generated, and allow people to enter a block of text to search the database. It wouldn’t beat determined users but it would at least catch the unaware.

Changing every 10th word defeats that strategy but doesn't defeat a cryptographic bias.

Also the cost of storing every paragraph hash might eventually add up even if at the moment it would be negligable compared to the generation cost.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#97

Post by klabb3 »

mercurialsolo wrote: post #68

I wonder why we need this very thing of AI generated. It's a luddite view of AI. Much like the need to distinguish between handcrafted versus machined products - is there a real utility to knowing this? For educators looking at evaluating students, essays and the like - we possibly need different ways of evaluation rather than on written asynchronous content for communicating concepts and ideas.

I believe you’re exactly right. It would be similar to detecting that math homework used wolfram alpha or even a calculator.

Re: OpenAI shuts down its AI Classifier due to poor accuracy

#99

Post by VeninVidiaVicii »

chakintosh wrote: post #21

This tool's been fueling tons of false accusations in academia. Wife is doing her PhD and she often tells me stories about professors falsely accusing students of using ChatGPT.

Eh I am doing my PhD and I use ChatGPT all the time!
Post reply on HN