Turing Test Success • Hacker News Board

Re: Turing Test Success

#81

Post by andreasvc » Sun, Jun 08, 2014, 10:21 PM UTC

mcguire wrote: post #30

skywhopper wrote: post #18

Where did the 30% requirement come from? Sounds like something the contest organizers added to make it possible to "pass" without fooling 2/3 of the judges. Using a young teenager as a character also seems like a cheat unless they had other 13-year olds to be the judges. The character needs to be a peer to the judges. Most 13 year olds behave oddly enough in the opinion of most adults that it's got to be easier to cr…

30% of the time, five minute conversations, a "simulated" 13-year old? At least "the conversations were unrestricted". The Turing test is not a test . Instead, it is an operational definition of "intelligence", a very slight formalization of the idea that something is intelligent if it seems to be intelligent . As a test, it obviously has to have some kind of limits like this "competition", but as soon as you put lim…

The Turing test is not a definition of intelligence. Turing explicitly suggests the test in order to replace the question of the definition of intelligence. The test is most definitely a test, in the sense that it provides an estimate, rather than a clear cut definition.

While the limits in this case were admittedly rather strong, this does not form a fundamental objection to the test. The bar can be set progressively higher.

Re: Turing Test Success

#82

Post by andreasvc » Sun, Jun 08, 2014, 10:35 PM UTC

acchow wrote: post #37

fchollet wrote: post #33

The Turing test is to our understanding of intelligence what sleight of hand is to our understanding of physics. Tricking people, as a goal, is not conductive to science. A researcher claiming to have passed the Turing test instantly discredits himself as a prestidigitator looking for PR buzz. The present article is a textbook example of this. As a side note, if you are focusing on disembodied, language-based human-l…

Your opinion is not universally accepted. (Perhaps not even mainstream?). There's an element of behaviorism which stretches back to Descartes - how can you know that I exist? That I think? You can only observe through my behavior; that my behavior mimics yours. How then can we judge machines any other way?

His opinion is actually a mainstream opinion in AI, see for example the textbook by Russell & Norvig. Descartes was the very opposite of a behaviorist, namely a rationalist. I do agree that we don't have anything that is clearly better than the Turing test, but this just goes to show how far we still have to go.

Re: Turing Test Success

#83

Post by e12e » Sun, Jun 08, 2014, 10:40 PM UTC

Between this and the Ars Technica article[1], I'm still confused: Was this a regular Turing test? Who was the humans that the machines tested against? As far as I recall, the model is two participants, one human, one machine -- the judges communicate with each through writing -- and if the machine "tests" as human more than 30% of the time, it's considered a "win" at the imitation game (the machine has successfully imitated being human). Both the machine and the human are supposed to try to appear human.

(And this is extended from another form of the imitation game, where the goal is to imitate being male, where participants are male and female)

Have anyone been able to find any more concrete information (and perhaps some transcripts)? If not I hope someone will set up a new test, and invite "Eugene" to participate.

[1] http://arstechnica.com/information-technology/2014/06/eugene...

[edit: We may be given some hints from the wikpedia article on the turing test: https://en.wikipedia.org/wiki/Turing_test#Imitation_Game_vs....

"Huma Shah and Kevin Warwick, who organised the 2008 Loebner Prize at Reading University which staged simultaneous comparison tests (one judge-two hidden interlocutors), showed that knowing/not knowing did not make a significant difference in some judges' determination. Judges were not explicitly told about the nature of the pairs of hidden interlocutors they would interrogate. Judges were able to distinguish human from machine, including when they were faced with control pairs of two humans and two machines embedded among the machine-human set ups. Spelling errors gave away the hidden-humans; machines were identified by 'speed of response' and lengthier utterances." ]

Re: Turing Test Success

#84

Post by e12e » Sun, Jun 08, 2014, 10:44 PM UTC

lukeschlather wrote: post #59

codeulike wrote: post #43

When people imagine what a Turing Test conversation would look like, they frequently underestimate the conversation. I find Dennet's example of an imaginary Turing Test from Consciousness Explained to be a good counterexample: Judge: Did you hear about the Irishman who found a magic lamp? When he rubbed it a genie appeared and granted him three wishes. “I’ll have a pint of Guiness!” the Irishman replied and immediate…

That excerpt reads to me like writing, not conversation. Someone spent some time polishing it. I know people who can talk like that extemporaneously, but I'd wager 99% of native English speakers wouldn't pass if that's the bar.

Remember that the imitation game that forms the foundation for the Turing test pits males versus females, with the goal of the females pretending to be male. Allowing speech would normally reveal the males due to the voice being different - it was therefore suggested that the test be preformed in writing.

[edit: Ah, I had the details wrong, see https://en.wikipedia.org/wiki/Turing_test ]

Re: Turing Test Success

#85

Post by andreasvc » Sun, Jun 08, 2014, 10:48 PM UTC

thisjepisje wrote: post #62

ronaldx wrote: post #27

Earlier quoted context omitted.

You can also pass the test this way by having "generous" judges contributing to the 1/3, which is likely because the judges are not impartial: they are emotionally invested in being part of a positive result. I wonder how Kevin Warwick himself voted, for example. A more correct test (which admittedly doesn't cover this issue) would be to give each judge a conversation with one human and one computer, and for them to…

>to give each judge a conversation with one human and one computer, and for them to say which one they believe is the human. I always assumed this was exactly what the Turing test was about. Guess I was wrong.

This is indeed what the Turing test as originally proposed is about.

Re: Turing Test Success

#86

Post by andreasvc » Sun, Jun 08, 2014, 10:53 PM UTC

TillE wrote: post #34

mcguire wrote: post #30

Earlier quoted context omitted.

30% of the time, five minute conversations, a "simulated" 13-year old? At least "the conversations were unrestricted". The Turing test is not a test . Instead, it is an operational definition of "intelligence", a very slight formalization of the idea that something is intelligent if it seems to be intelligent . As a test, it obviously has to have some kind of limits like this "competition", but as soon as you put lim…

> Instead, it is an operational definition of "intelligence" Exactly. It's a straightforward formulation of what a strong AI would be capable of. It makes no sense to have a restricted Turing test that can be passed by a useless chatbot. It means absolutely nothing.

I assume you mean weak AI. And I don't see how you can dismiss the meaning of passing this test so easily. Although it's a trivial example, I can definitely see such chatbots being applied for spamming purposes, which by definition exploits hapless victims.

Re: Turing Test Success

#87

Post by andreasvc » Sun, Jun 08, 2014, 10:57 PM UTC

e12e wrote: post #83

Between this and the Ars Technica article[1], I'm still confused: Was this a regular Turing test? Who was the humans that the machines tested against? As far as I recall, the model is two participants, one human, one machine -- the judges communicate with each through writing -- and if the machine "tests" as human more than 30% of the time, it's considered a "win" at the imitation game (the machine has successfully i…

Adding random spelling errors and delays to responses should be one of the more trivial "improvements" to a chatbot.

Re: Turing Test Success

#88

Post by SilasX » Sun, Jun 08, 2014, 10:59 PM UTC

codeulike wrote: post #67

DanBC wrote: post #64

Earlier quoted context omitted.

Has Dennet never seen Facebook or Youtube?

Not when he wrote that (1991)

Ironically, in a later book (Darwin's Dangerous Idea, 1995) he mocks people who got duped by an Eliza program on a disconnected laptop, since "obviously" a computer must be physically connected to the wall in order to talk to the outside world.

Re: Turing Test Success

#89

Post by tritium » Sun, Jun 08, 2014, 11:00 PM UTC

mratzloff wrote: post #73

tritium wrote: post #24

Earlier quoted context omitted.

Generally it involves using "average people" [...] it should consist of computer science experts instead I don't agree. The prominent reason for "dumbing down" the judges follows the same reasoning behind decisions made regarding what constitutes "adequate" encryption. How can we gauge what will honestly happen out in the real world, today? Consider DES. It was deemed inadequate, but how to prove it? The EFF came up…

Since this chat bot passed the low bar set forth, do you believe that it is intelligent? Do stage magicians perform real magic simply because they fool the audience?

I believe it's practical to maintain an awareness that the bar is as absurdly low as it seems to be, and that well-developed deception may become the norm when interacting with online entities.

Do I believe that it's intelligent? Well, I wouldn't confer human rights to it. It doesn't carry the weight of emotional investment that a domestic pet might.

But that's the sort of thing to stay wary of. Average people becoming emotionally invested in silly things. People being tricked into carrying around an urn full of ashes, and believing that a convincing AI truly represents their late relatives, and similar sorts of tomfoolery.

Re: Turing Test Success

#90

Post by [deleted] » Sun, Jun 08, 2014, 11:09 PM UTC

SilasX wrote: post #88

codeulike wrote: post #67

Earlier quoted context omitted.

Not when he wrote that (1991)

Ironically, in a later book (Darwin's Dangerous Idea, 1995) he mocks people who got duped by an Eliza program on a disconnected laptop, since "obviously" a computer must be physically connected to the wall in order to talk to the outside world.

[deleted]