[deleted]
Ask HN: How would you build a ChatGPT detector?
101–109 of 109 posts
Re: Ask HN: How would you build a ChatGPT detector?
#102Realistically, the internet will be completely unreliable in 5 years. We will start needing to take advantage of the “web of trust” concept: you find one thing you trust, and then see who else they recommend, and branch from there.
This will be extremely painful though, and make discoverability impossible for new artists and writers.
Will we go back to the gatekeeper model of public exposure?
Will human work begin to be shunned?
Re: Ask HN: How would you build a ChatGPT detector?
#103It might be possible to use ChatGPT model itself to calculate probability (perplexity) of a given text according to the model.
Re: Ask HN: How would you build a ChatGPT detector?
#104A better question is: can you tell when a person or a computer is wrong in the first place...
Developing your own intuition for the world will help you better filter bad gossip and bad ai... ultimately to me if the AI is correct then it's welcome.
Re: Ask HN: How would you build a ChatGPT detector?
#105Earlier quoted context omitted.
> Overall, I think building a ChatGPT detector is a great idea, and I'm confident that with the right approach, it could be a valuable tool for anyone who uses chat platforms. I think that hollowly summing up and reiterating a point of the whole text in the last sentence might be a good signal to differentiate Open AI and humans. AI seems to be doing that on nearly all creative responses.
It ia a fairly widely taught grade school technique that many people never get beyond that is probably heavily represented in its training corpus.
But people usually don't summarize lists of 2-3 short paragraphs, repeating contents of these paragraphs[1]. Such comments can be just found with a stupid regexps like /(One|First|The first) [^\n]{100,600}\n\n(Next|Second|Another)[^\n]{100,600}\n\n(Overall|In summary)[^\n]{100,600}/. Also it is not always needed to detect ChatGPT from a single comment. If there is just few definite pattern that only machines use (like in chess), then it is enough to flag accounts.
Re: Ask HN: How would you build a ChatGPT detector?
#106Lol, it wouldn't be that hard to build an AI that could recognize ChatGPT's writing. I mean, it's not like ChatGPT is producing some super unique and creative language or anything. It's just spitting out the same old generic responses to prompts. If you want to build an AI that could accurately recognize ChatGPT's writing, just train it on a bunch of examples of ChatGPT's responses and it'll be able to pick out the c…
ChatGPT likes to repeat words that were mentioned in the prompt. Especially this is noticeable when there are no synonyms. In your short paragraph, "ChatGPT" is mentioned 5 times (!). So in some hypothetical situation, where Stackoverflow admins decide to write a detector, they could use it. And if bot owner decide to skip detectable messages (maybe by using a discriminator for updating weights), then the quality of answers will decrease even more.
Re: Ask HN: How would you build a ChatGPT detector?
#107The top answers here seem to be funny, but lazy. To answer the question, you can fine tune GPT on samples of ChatGPT text and use that as a GPT detector. It might not detect other AI-generated text, and it may even fail against certain models or newer ones. The idea is that it averages out into a certain tone fingerprint. It wouldn't be very difficult, but it also won't be very stable.
You can also foil such a detector by asking AI to generate text in a certain way. Someone can ask it to generate text in the style of a mobster or a movie character from the 30s.
Re: Ask HN: How would you build a ChatGPT detector?
#108Your point is well taken. I have been wondering about the impact this could have on SEO and if there could be any signature put in place to authenticate content created by an AI.
Re: Ask HN: How would you build a ChatGPT detector?
#109Earlier quoted context omitted.
I suspect that in time, this will only accelerate the degree to which AI and human-authored text are indistinguishable from each other.
I've already sent text to customers 100% written by AI. Ethically dubious in a commercial setting perhaps, but higher quality text than I would be able to produce myself. I asked OpenAI and it said: It is not necessarily unethical to send customers text generated by AI, but it depends on the context and the specific situation. For example, if the text is being used to deceive or mislead customers, then it would be un…
I use LanguageTools as a grammar checker, and the last update included suggestion by AI. It’s quite nice as a way to explore alternate versions of my writing. But some of them don’t reflect what I’m trying to say. I don’t think I will ever trust an AI without a human as a filter.