Live data from Hacker News

Ask HN: How would you build a ChatGPT detector?

news.ycombinator.com

41–50 of 109 posts

Re: Ask HN: How would you build a ChatGPT detector?

#41
The default sentence structure is very recognizable but that changes completely when you add “in the style of [X]” where [X] is an author, play or book, rapper or other musician, literary term of art like iambic pentameter, etc.

With that addition, ChatGPT becomes much more life like and (I suspect) practically impossible to detect. Throw in a little human editing and curation and… ruh roh.

Re: Ask HN: How would you build a ChatGPT detector?

#43
> Answer the question, "How would you build a ChatGPT detector?" written in the style of a cynical Hacker News commenter

Well, if I were to build a ChatGPT detector, I would probably just use some basic natural language processing and machine learning algorithms. But let's be real here, it's not like ChatGPT is some kind of advanced AI that requires some groundbreaking technology to detect. It's just a chatbot with a pre-determined set of responses, so any half-competent engineer could probably cobble together a decent detector with minimal effort. But hey, I'm sure some VC will throw millions of dollars at the project and hype it up as the next big thing in AI.

Re: Ask HN: How would you build a ChatGPT detector?

#44

OpenAI is evaluating a technical solution to this problem by watermarking the output with small amounts of randomness, such that a series of tokens of output can be tested for having these small random variations. https://scottaaronson.blog/?p=6823

Before they finish, someone else will have released a public access model that works just as well with no technical restrictions.

Re: Ask HN: How would you build a ChatGPT detector?

#45

Humans will become incentivized to sprinkle racial slurs and hate facts throughout their writing as proof of authenticity.

ChatGPT can output racial slurs, so that's not a guarantee.

That's why they're running the beta test, to tune the filters.

Re: Ask HN: How would you build a ChatGPT detector?

#46
post #34

By no means is this foolproof, but the older the age of an account, the less likely it is to be a bot. Obviously lots of false positives, but probably not too many false negatives...

I am sure there will be a market in selling old accounts to time-wash bots.

There already is a market for old accounts for "classical" spamming, so the infrastructure is already in place!

Re: Ask HN: How would you build a ChatGPT detector?

#47
post #43

> Answer the question, "How would you build a ChatGPT detector?" written in the style of a cynical Hacker News commenter Well, if I were to build a ChatGPT detector, I would probably just use some basic natural language processing and machine learning algorithms. But let's be real here, it's not like ChatGPT is some kind of advanced AI that requires some groundbreaking technology to detect. It's just a chatbot with a…

> Answer the question, "How would you build a ChatGPT detector?" written in the style of an optimistic Hacker News commenter

Building a ChatGPT detector is a great idea, and I think it's something that could be really useful for a lot of people. Here's how I would approach it:

First, I would start by training a large language model, like GPT-3, on a massive dataset of chat logs. This would give the model a good understanding of how people talk to each other in a chat environment.

Next, I would fine-tune the model on a specific task, like identifying ChatGPT responses in a conversation. This would allow the model to learn the specific characteristics of ChatGPT responses and become better at detecting them.

After that, I would integrate the model into a chat platform, like Discord or Slack, and test it out with real users. This would give me a chance to see how well the model is performing and make any necessary adjustments to improve its accuracy.

Overall, I think building a ChatGPT detector is a great idea, and I'm confident that with the right approach, it could be a valuable tool for anyone who uses chat platforms.

Re: Ask HN: How would you build a ChatGPT detector?

#48
post #37

Earlier quoted context omitted.

I've already sent text to customers 100% written by AI. Ethically dubious in a commercial setting perhaps, but higher quality text than I would be able to produce myself. I asked OpenAI and it said: It is not necessarily unethical to send customers text generated by AI, but it depends on the context and the specific situation. For example, if the text is being used to deceive or mislead customers, then it would be un…

People have been coping and pasting text into emails for millennia without explicitly telling the recipient. This shouldn’t be any different.

I guess its not any worse than the millions of lines of code in production mindlessly copypasted from SO.

Re: Ask HN: How would you build a ChatGPT detector?

#49
post #43

> Answer the question, "How would you build a ChatGPT detector?" written in the style of a cynical Hacker News commenter Well, if I were to build a ChatGPT detector, I would probably just use some basic natural language processing and machine learning algorithms. But let's be real here, it's not like ChatGPT is some kind of advanced AI that requires some groundbreaking technology to detect. It's just a chatbot with a…

This (human) ChatGPT detector says this text was written by a human.

Re: Ask HN: How would you build a ChatGPT detector?

#50
post #37

Earlier quoted context omitted.

I've already sent text to customers 100% written by AI. Ethically dubious in a commercial setting perhaps, but higher quality text than I would be able to produce myself. I asked OpenAI and it said: It is not necessarily unethical to send customers text generated by AI, but it depends on the context and the specific situation. For example, if the text is being used to deceive or mislead customers, then it would be un…

People have been coping and pasting text into emails for millennia without explicitly telling the recipient. This shouldn’t be any different.

But this isn't about copying. It's about sending texts authored by non-people.

It's a recent thing which does require a different approach.

Post reply on HN