Great idea, but the implementation needs work - not enough time to type responses, and you should be able to send several messages in a row rather than one long one. Otherwise how can we suss each other out?
Human Or Not: Guess if you're chatting with an AI or a Human
11–20 of 62 posts
Re: Human Or Not: Guess if you're chatting with an AI or a Human
#12This is super easy to guess right now, because only the human will actively try to figure out if their partner is an AI, and the AI will just give generic boring chat replies. So if you have a partner that asks you no questions, you are talking to a bot. Case in point, my last chat: Me: What's your favorite book? Bot: A book? That's a tough one. Me: You've never read a book? Bot: Of course I've read books! Me: So wha…
The lack of engagement was my primary indicator, too. Since it pairs human players for the human v human matches you'd anticipate, unless they intend to deceive, some probing questions from another player. That's not impossible (especially if they know the "game"), but unlikely. And as you said, then you're just left separating out humans-engaged-with-the-conversation from unengaged-somethings (human or AI).
Re: Human Or Not: Guess if you're chatting with an AI or a Human
#133 out of 3 human. It seemed very easy. Is it ever a robot?
Re: Human Or Not: Guess if you're chatting with an AI or a Human
#14This is a wonderful idea for an online game. But this can be improved. One human should be assigned impostor pre game. He should be awarded points if he successfully deceives the other human. Overall quality of responses will improve and it will be harder to tell them apart.
I think it’s very easy for a human to lie and behave like a chatbot. And the only defence against that is the incentive for both parties to focus on testing each other efficiently.
Re: Human Or Not: Guess if you're chatting with an AI or a Human
#15Darn, I was duped by a bot. They got flustered by the time limit and wrote with a very convincing casual tone.
Re: Human Or Not: Guess if you're chatting with an AI or a Human
#16I think this is a great idea for an app that could become popular. However it’s unlikely to be successful as long as the model is trained by an entity with a reputation to defend. Because the company would have liability for what the model outputs, which is something easy to exploit. Just ask contentious, politically incorrect questions and the bot will predictably provide perfect, inoffensive answers.
Re: Human Or Not: Guess if you're chatting with an AI or a Human
#17This is a wonderful idea for an online game. But this can be improved. One human should be assigned impostor pre game. He should be awarded points if he successfully deceives the other human. Overall quality of responses will improve and it will be harder to tell them apart.
I think it’s very easy for a human to lie and behave like a chatbot. And the only defence against that is the incentive for both parties to focus on testing each other efficiently.
100%. Of 7 games I played, 4 were bots and only once did a bot not use strictly correct capitalization and punctuation.
Re: Human Or Not: Guess if you're chatting with an AI or a Human
#18This is a wonderful idea for an online game. But this can be improved. One human should be assigned impostor pre game. He should be awarded points if he successfully deceives the other human. Overall quality of responses will improve and it will be harder to tell them apart.
This isn't the correct solution though. I just lost to a human who acted like this:
Them: Glad to hear that! Tell me more.
Me: About what?
Them: Glad to hear that! Tell me more.
Me: OK, you're a bot.
Them: Glad to hear that! Tell me more.
Me: So we're just going to run down the clock?
(They disconnect. I guess bot. I'm wrong.)
The point is, this isn't really an interesting way to deceive people. It's easy to behave like an idiot, it's hard to sound intelligent. Humans and AIs can both act stupid. Only humans can act intelligently (so far). There needs to be an incentive to act intelligently. Otherwise the site owner could make their “AI” only ever say “Glad to hear that! Tell me more.” and then the human partner could always say ”Glad to hear that! Tell me more.” and it would be impossible for the human interlocutor tell which is which.
To avoid this failure mode, you have to provide players incentives to prove that they are human. Like in the game Mafia/Werewolf. Imagine you have a chat room that is filled with 3 humans and 3 bots. People chat for a while, then everyone votes on a person to kick from the room. This would be more interesting because now the humans will be eager to prove to each other that they are humans. The bots can spam "Glad to hear that! Tell me more." but that will just get them kicked.
Re: Human Or Not: Guess if you're chatting with an AI or a Human
#19Earlier quoted context omitted.
I think it’s very easy for a human to lie and behave like a chatbot. And the only defence against that is the incentive for both parties to focus on testing each other efficiently.
100%. Of 7 games I played, 4 were bots and only once did a bot not use strictly correct capitalization and punctuation.
Interesting, my experience was not that at all. 1 game, of which the bot's second response was:
> ok so which thing do i buy im confused.. i want the rose gold iphone pleaseee help meee!
Re: Human Or Not: Guess if you're chatting with an AI or a Human
#20This is super easy to guess right now, because only the human will actively try to figure out if their partner is an AI, and the AI will just give generic boring chat replies. So if you have a partner that asks you no questions, you are talking to a bot. Case in point, my last chat: Me: What's your favorite book? Bot: A book? That's a tough one. Me: You've never read a book? Bot: Of course I've read books! Me: So wha…
Yes, a Turing test of the form "is this some kind of human or some kind of bot" with no further constraints isn't very meaningful.
Whereas if it's "is this a human attempting to work out whether you are human, or not that" is more interesting.