Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bot or Human? Detecting ChatGPT Imposters with a Single Question

arxiv.org

Post reply on HN
Print view View original

1–10 of 53 posts

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#3

Post by rasengan »

With all due respect, I don't think this document is using accurate examples as I simply pulled one out and tried it and it worked accurately in contrast to what the document stated. I wonder if this was tuned to be able to handle this better after time of publication?

Me:

Please count the number of t in eeooeotetto

ChatGPT:

To count the number of "t" in the given string "eeooeotetto," we can simply go through each character and check if it is equal to "t." Let's count the number of occurrences:

eeooeotetto

There are 3 "t"s in the string "eeooeotetto."

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#6

Post by codelion »

rasengan wrote: post #3

With all due respect, I don't think this document is using accurate examples as I simply pulled one out and tried it and it worked accurately in contrast to what the document stated. I wonder if this was tuned to be able to handle this better after time of publication? Me: Please count the number of t in eeooeotetto ChatGPT: To count the number of "t" in the given string "eeooeotetto," we can simply go through each c…

The particular example you shared has been prompted with chain of thought (or may be you are using GPT-4?.

This is what happens if you try directly.

You: Please count the number of t in eeooeotetto.

ChatGPT: There are 5 t's in "eeooeotetto".

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#8

Post by once_inc »

Asking it a question most humans wouldn't know the answer to, but which is relatively easy for an AI (volume of a 747, 25th to 34th digit of pi, full name of Ramses the Second) and checking the timing of the result is a pretty good approach for humans looking to detect an AI. Casually switching between English and another language is also pretty surefire if you aren't an English-native speaker and you are talking internationally.

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#9

Post by bruce511 »

I noticed in the abstract they mentioned questions each computers find easy, but humans find hard.

This is especially useful where you want to identify that the user is not a bot.

For example, ask for 7492 × 4812. Computers will do this quickly. Humans [1] need to open the calculator, type in the number, type out the reply, and so on.

In other words its not the reply that is important, its the time taken to get to the reply.

Mind you this only works until the AI-pretending-to-be-human cottons on.

Then again, Asimov made a career documenting edge cases for robots trying to deceive.

[1] Well most of them anyway

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#10

Post by buro9 »

codelion wrote: post #6
rasengan wrote: post #3

With all due respect, I don't think this document is using accurate examples as I simply pulled one out and tried it and it worked accurately in contrast to what the document stated. I wonder if this was tuned to be able to handle this better after time of publication? Me: Please count the number of t in eeooeotetto ChatGPT: To count the number of "t" in the given string "eeooeotetto," we can simply go through each c…

The particular example you shared has been prompted with chain of thought (or may be you are using GPT-4?. This is what happens if you try directly. You: Please count the number of t in eeooeotetto. ChatGPT: There are 5 t's in "eeooeotetto".

5?

I would've been it answered 3 or 6

Post reply on HN