Live data from Hacker News

DetectGPT: Zero-Shot Machine-Generated Text Detection

ericmitchell.ai

21–30 of 71 posts

Re: DetectGPT: Zero-Shot Machine-Generated Text Detection

#21

Is there really something of value to be had by fighting a war against the coming tide of AI that can spit out whatever you ask for? To me it looks like old world professors calling the council because the abacus is under threat by the calculator, and we need to ensure the students are absolutely not using the calculator for their abacus studies. This all being despite the fact that society at large is moving as fast…

It is valuable for SPAM filters, and other fraudulent activity. If you get an e-mail from someone you know and it says "I need itunes gift cards right away!" most people know that is a scam. But if you get an e-mail that appears to be following up on a previous conversation, you are much more likely to click on a link in that e-mail.

Re: DetectGPT: Zero-Shot Machine-Generated Text Detection

#23

From a technical standpoint, this is interesting. Like all other methods I’ve seen, it requires access to the original model logits, so it can’t be used in closed models like GPT-3 except by OpenAI themselves. But I would classify this into the category of dangerous research. There is a conflict of interest: overstating the effectiveness leads less technical people to give undue trust to what is ultimately a statisti…

You can get all the necessary log probabilities from the GPT-3 API.

Seems to me you might be able to guess without the weights by manually changing a word or two and seeing how much gpt wants to change.

Re: DetectGPT: Zero-Shot Machine-Generated Text Detection

#24

Is there really something of value to be had by fighting a war against the coming tide of AI that can spit out whatever you ask for? To me it looks like old world professors calling the council because the abacus is under threat by the calculator, and we need to ensure the students are absolutely not using the calculator for their abacus studies. This all being despite the fact that society at large is moving as fast…

It is valuable for SPAM filters, and other fraudulent activity. If you get an e-mail from someone you know and it says "I need itunes gift cards right away!" most people know that is a scam. But if you get an e-mail that appears to be following up on a previous conversation, you are much more likely to click on a link in that e-mail.

I doubt spammers will be the only ones leveraging AI generated text. I'm sure legit marketing people and even administrators and support staff will benefit from using AI written responses. So you're going to filter out valuable emails as well.

Re: DetectGPT: Zero-Shot Machine-Generated Text Detection

#25

I can't judge the capabilities of this tool, but I can't help but be skeptical on the mid-long term prospects. The fakes (text, audio, video) will improve to the point of being virtually unrecognizable from real material, no matter how good your fake detection is.

Can some chess players detect when an opponent is using stockfish? Yes, if they’re Magnus Carlsen and memorized a particular line of play. This is a bit like that. But yes, I suppose when these tools become so fluent that they basically never need to use the same pattern twice, we’re toast.

Re: DetectGPT: Zero-Shot Machine-Generated Text Detection

#26

The approach to feed text through a language model and then obtain some kind of vector/hash from it will be leveraged to fingerprint writing and track people.

In addition, this can feed a sort of “recaptcha” loop that trains AI to notice AI and create new text less likely to be noticed…

Re: DetectGPT: Zero-Shot Machine-Generated Text Detection

#27

> This approach, which we call DetectGPT, does not require training a separate classifier, collecting a dataset of real or generated passages, or explicitly watermarking generated text. It uses only log probabilities computed by the model of interest and random perturbations of the passage from another generic pre-trained language model (e.g, T5). Very interesting, though the passage above makes me wonder how robust…

[deleted]

Re: DetectGPT: Zero-Shot Machine-Generated Text Detection

#28
> For example, students may use LLMs to complete written assignments, leaving instructors unable to accurately assess student learning.

I'm one student using GPT to write my written assignments, and I cannot understand what is the issue with that. Even though it enhances the quality of my text, I cannot just copy, paste, submit and profit. I still have to understand what is being written. My workflow is normally the following:

- I ask for an outline to the essay. I use this outline as guide, remove some suggestion, add others that I think are important, rephrase some. I could as well google for similar written essays and do the same.

- Then I ask for the topics, please describe X, Y.. I read it, as a regular source. I understand it, and rewrite it. I add stuff, remove stuff, add references. Normally I endup rewriting everything

- Push the text through Grammarly, rephrase some stuff, check for plagiarism and submit it. I have to read and re-read it, many times.

I have the impression that I've got much more effective avoiding the mechanical part of the task of writing. In another hand my course has some stupid enforcement like 15k words/essay. So they are almost asking me to do some bullshit around the topic instead of going straight to the point. I simply automate the burden. As any software developer would do. I however study philosophy.

So i guess, it will help the universities and education institutions to rethink their traditional way of evaluate written assignments, which probably will be positive. It helps me definitely to write better.

Re: DetectGPT: Zero-Shot Machine-Generated Text Detection

#29

Is there really something of value to be had by fighting a war against the coming tide of AI that can spit out whatever you ask for? To me it looks like old world professors calling the council because the abacus is under threat by the calculator, and we need to ensure the students are absolutely not using the calculator for their abacus studies. This all being despite the fact that society at large is moving as fast…

> I'm still gonna chose my AI consultant over you in 5-10 years.

AI will never replace experts.

Someone still has to curate the firehose of information returned. What's the point then?

AI can only interpolate a response from internet sources, and claims of plagiarism are already coming hard. The internet was never a good source of expertise.

Re: DetectGPT: Zero-Shot Machine-Generated Text Detection

#30

From a technical standpoint, this is interesting. Like all other methods I’ve seen, it requires access to the original model logits, so it can’t be used in closed models like GPT-3 except by OpenAI themselves. But I would classify this into the category of dangerous research. There is a conflict of interest: overstating the effectiveness leads less technical people to give undue trust to what is ultimately a statisti…

You can get all the necessary log probabilities from the GPT-3 API.

Hmm seems you may be right. You can get the top 5 out of the box but you’ll need to file for an exception to get any more than that.
Post reply on HN