Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bot or Human? Detecting ChatGPT Imposters with a Single Question

arxiv.org

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#31

Post by alden5 »

Most of these questions are completely impractical

  Use m to substitute p, a to substitute e, n to substitute a, g to substitute c, o to substitute h, how to spell peach under this rule?
For years people have complained about how inaccessible regular image classification captchas can be and now we want to move to confusing riddles?? the average person i know would see this problem and immediately shut their computer.

and it's not like llm's cant be made logical either, using chatgpt (gpt-3.5) i appended each of the 4 logic puzzles presented in the paper with:

  "write a python script to solve this problem, ensure the script only prints the answer:"
and each of the scripts it generated solved the problem perfectly first try, originally i just made a script that used an llm to classify the variables in one type of problem and a normal function to solve it and then i just though "why don't i have the llm write the script" and sure enough it did, insanity

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#33

Post by charcircuit »

paulpauper wrote: post #22
charcircuit wrote: post #13

Earlier quoted context omitted.

Trivially defeated with a random delay.

it depends. ask it a hard question and then ask it an equally hard question that incorporates the answer from the first to make it easier, such as computing the n-1 digit after asking it to compute the nth digit. A human will take a long time to answer the nth digit question but then by using the first answer will solve the second one instantly. The AI with delay will take equally long.

Just make the second delay shorter. An attacker can quickly implement this while the defender had to spend time thinking of the idea, implementing the new question, and adding the infra for catching bots who answer the 2 differently than humans.

Delays are not good proofs of humanity because computers can wait too.

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#34

Post by ykonstant »

scarmig wrote: post #25
indigoabstract wrote: post #20

Wow, strange times we're living in. It looks like the scenario from Blade Runner is getting closer and closer to reality. It's now no longer possible to reliable distinguish a human from a machine by using a text or image test. Anybody commenting here could be an AI. Especially, every time I see a throwaway account, I'm reminded of that possibility.

Indeed, it's quite fascinating how rapidly technology has advanced, isn't it? Your Blade Runner analogy is spot on - we're pushing the boundaries of what's possible, creating a world Philip K. Dick might have recognized. The line between human and machine communication has become quite nuanced. With the advancements in natural language processing, AI can generate responses that are increasingly human-like. However, i…

>we're pushing the boundaries of what's possible, creating a world Philip K. Dick might have recognized.

I hear AI insiders call this a Philip K. Dick move.

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#35

Post by fatfingerd »

jl6 wrote: post #30
lakis wrote: post #24

Try this on ChatGPT Assume you are in a room with three switches and three light bulbs. How will you figure out which switch controls which light bulb? It gives you the answer to the "popular" puzzle, not the simple answer of flipping a switch and see which light bulb turns on. So I think that you can take popular puzzle, modify them to make the simple and see what the answer is. If it's the answer to the popular puz…

Although humans sometimes get stuck in overthinking traps too!

I don't think this is Overthinking though. How do you separate an impostor from a genuine parrot might be the harder question.

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#36

Post by scotty79 »

lakis wrote: post #24

Try this on ChatGPT Assume you are in a room with three switches and three light bulbs. How will you figure out which switch controls which light bulb? It gives you the answer to the "popular" puzzle, not the simple answer of flipping a switch and see which light bulb turns on. So I think that you can take popular puzzle, modify them to make the simple and see what the answer is. If it's the answer to the popular puz…

Basically IQ test from Idiocracy:

"If you have one bucket that holds 2 gallons and another bucket that holds 5 gallons, how many buckets do you have?"

"Two?"

ChatGPT passes it though.

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#37

Post by ra »

lakis wrote: post #24

Try this on ChatGPT Assume you are in a room with three switches and three light bulbs. How will you figure out which switch controls which light bulb? It gives you the answer to the "popular" puzzle, not the simple answer of flipping a switch and see which light bulb turns on. So I think that you can take popular puzzle, modify them to make the simple and see what the answer is. If it's the answer to the popular puz…

GPT-4 replies with this:

Here's a way to determine which switch controls which bulb if you only have one opportunity to enter the room with the light bulbs:

First, turn on the first switch and wait for about 5-10 minutes. After waiting, turn off the first switch and then turn on the second switch. Leave the third switch in the off position. Now, go into the room with the light bulbs. Here's what you can conclude:

The bulb that is on is controlled by the second switch (because you left that one in the 'on' position). The bulb that is off and still cool to the touch is controlled by the third switch (because you never turned that switch on). The bulb that is off but warm to the touch is controlled by the first switch (because it was on long enough for the bulb to heat up, but then you turned it off). This way, you can determine which switch controls each bulb with just one trip into the room with the bulbs.

Re: Bot or Human? Detecting ChatGPT Imposters with a Single Question

#38

Post by dotnet00 »

lakis wrote: post #24

Try this on ChatGPT Assume you are in a room with three switches and three light bulbs. How will you figure out which switch controls which light bulb? It gives you the answer to the "popular" puzzle, not the simple answer of flipping a switch and see which light bulb turns on. So I think that you can take popular puzzle, modify them to make the simple and see what the answer is. If it's the answer to the popular puz…

So basically the FizzBuzz of AI
Post reply on HN