Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Conversation with “Eugene Goostman”

scottaaronson.com

Post reply on HN
Print view View original

51–60 of 81 posts

Re: My Conversation with “Eugene Goostman”

#51

Post by DanBC »

anonymfus wrote: post #41
NAFV_P wrote: post #26

Earlier quoted context omitted.

> I would get tired of typing "Invalid syntax." Good point, but put yourself in the computer's shoes. If I were an artificial computer, talking to a human would drive me insane.

*> If I were an artificial computer Opposed to natural computers like humans? That is very interesting terminology.

That's terminology straight from Turing's paper introducing the game.

Computers were people back then, as well as digital computers. And he talks about cloning a human not counting as a win for an intelligent machine.

Re: My Conversation with “Eugene Goostman”

#52

Post by fchollet »

ecopoesis wrote: post #45
fchollet wrote: post #10

There is nothing about Eugene that is useful to the advancement of actual AI, to our understanding of intelligence. Advanced chatbots are a parlor trick, the same way that rabbit and hat tricks do not help us develop actual teleportation systems, however clever and convincing they might seem to credulous people. The difference between a trick and a technology, of course, is usefulness. Can your software do something…

If the machine can "deceive" you into thinking it's doing work, then it's doing work. It doesn't matter if there's a man who speaks Chinese in the room, or if there's a bunch of rulebooks about Chinese, they're functionally equivalent. http://en.wikipedia.org/wiki/Chinese_room

The difference between an employee doing work and one deceiving you into thinking they're doing work is simple: one produces value, the other does not. Here, no useful information can be extracted from this Eugene chatbot. It cannot perform any useful task a human could perform or answer any useful question a human could answer. Which is what I was talking about when I mentioned usefulness.

As a side note, I recently made a question answering engine which, although not very advanced and certainly not attempting to pass for a human, can provide you with useful information when asked general knowledge questions:

http://www.sphere-engineering.com/blog/quickanswers-io-seman...

The Chinese room experiment is not a relevant reference here, because this Eugene chatbot is in no way "functionally equivalent" to a human being. It is not even attempting to implement intelligence.

Re: My Conversation with “Eugene Goostman”

#53

Post by ThomPete »

fchollet wrote: post #52
ecopoesis wrote: post #45

Earlier quoted context omitted.

If the machine can "deceive" you into thinking it's doing work, then it's doing work. It doesn't matter if there's a man who speaks Chinese in the room, or if there's a bunch of rulebooks about Chinese, they're functionally equivalent. http://en.wikipedia.org/wiki/Chinese_room

The difference between an employee doing work and one deceiving you into thinking they're doing work is simple: one produces value, the other does not. Here, no useful information can be extracted from this Eugene chatbot. It cannot perform any useful task a human could perform or answer any useful question a human could answer. Which is what I was talking about when I mentioned usefulness. As a side note, I recently…

You keep assuming something about the turing test that is simply not the case. The Turing test has nothing to do with usefulness it doesn't even have to be correct. It just has to fool you into believing it's human.

You are talking about your own metrics, fine but it's not what the test is about. I too hear Searles Chinese Room argument in your basic rant.

And yes of course there are many useful things that can come out of something like this such as phishing attacks, hacking, honeypots even toys for kids.

Re: My Conversation with “Eugene Goostman”

#54

Post by matnewton85 »

marcus_holmes wrote: post #39
matnewton85 wrote: post #35

Earlier quoted context omitted.

Ok, I think this is just a perspective thing. You're (most likely) coming from a CS background, I come from a user testing background. If someone sits me down and says "use my site, have a conversation with a 13 year old Ukrainian".. I start having a conversation with a 13 year old Ukrainian. Someone sits a CS major down with Eugene, the 13 year old Ukrainian, he'll drop references to AI reserch from the 60s. Somethi…

but these bots won't cope with not being able to answer, because the human response (to learn) isn't open to them. If you sit down with a human and start talking about something they don't know, then usually (in a situation free from conflict or other emotional prompts) their curiosity prompts them to start asking questions and building a mental model of the subject that prompts further questions and they learn about…

Great point.

Re: My Conversation with “Eugene Goostman”

#55

Post by jpindar »

Houshalter wrote: post #34
V-2 wrote: post #9

Clever as this is, how could it really fool anyone (aware of the existence of bots)?

Well this is a much older version of the bot I believe. Additionally most people do not ask adversarial questions, but rather try to have a normal conversation, which bots are quite good at. You also have to have prior knowledge about how chatbots work in order to know what kinds of things it is bad at (e.g. common sense knowledge about the world, which most people wouldn't think to ask about or would come up in conv…

The normal conversations I have with people do not consist primarily of them trying to change the subject.

Re: My Conversation with “Eugene Goostman”

#56

Post by jmckib »

davidgerard wrote: post #7

The entire Turing test claim is Kevin Warwick's media hype. He issues a press release to this effect approximately annually. Basically, the way to tell if a claim related to artificial intelligence is rubbish is to search for the name "Kevin Warwick". This is what I wrote for his 2008 effort: http://newstechnica.com/2008/10/13/turing-test-won-with-arti...

It seems that Kevin Warwick is well known for doing this sort of thing, so I wonder why it isn't evident on his wikipedia page http://en.wikipedia.org/wiki/Kevin_Warwick

Re: My Conversation with “Eugene Goostman”

#57

Post by Houshalter »

jpindar wrote: post #55
Houshalter wrote: post #34

Earlier quoted context omitted.

Well this is a much older version of the bot I believe. Additionally most people do not ask adversarial questions, but rather try to have a normal conversation, which bots are quite good at. You also have to have prior knowledge about how chatbots work in order to know what kinds of things it is bad at (e.g. common sense knowledge about the world, which most people wouldn't think to ask about or would come up in conv…

The normal conversations I have with people do not consist primarily of them trying to change the subject.

Neither does a "normal" conversation with the bot. It only tries to change it when you start probing for weaknesses and going off script.

Re: My Conversation with “Eugene Goostman”

#59

Post by eridius »

Timmmmbob wrote: post #12
jqm wrote: post #11

Based on this article, if 33% of the judges were fooled maybe some better judges are in order. It seems there are two ways to beat the Turing test. Come up with very clever algorithms or (probably easier) get some less discerning judges. The whole thing reminds me of this: http://www.cleverbot.com/ Not Turing convincing, but I found it fun.

Yeah the Onion article writes itself: "Turing test proves that 30% of judges are actually computers." A recent Turing test at Reading University backfired after it was discovered that nearly a third of the apparently human judges were discovered to be less intelligent than a ZX Spectrum. Suspicions were raised when they were unable to carry on sophisticated conversations with the programs being tested and instead nat…

Relevant XKCD: http://xkcd.com/329/

Re: My Conversation with “Eugene Goostman”

#60

Post by lotharbot »

ecopoesis wrote: post #45
fchollet wrote: post #10

There is nothing about Eugene that is useful to the advancement of actual AI, to our understanding of intelligence. Advanced chatbots are a parlor trick, the same way that rabbit and hat tricks do not help us develop actual teleportation systems, however clever and convincing they might seem to credulous people. The difference between a trick and a technology, of course, is usefulness. Can your software do something…

If the machine can "deceive" you into thinking it's doing work, then it's doing work. It doesn't matter if there's a man who speaks Chinese in the room, or if there's a bunch of rulebooks about Chinese, they're functionally equivalent. http://en.wikipedia.org/wiki/Chinese_room

> "If the machine can "deceive" you into thinking it's doing work, then it's doing work."

That depends on how long the deception lasts. Do you think it's doing work for the first few seconds of interaction, for a few minutes, or do you never figure out the deception?

From what I understand of the Turing test, the deception has to last at least for an entire conversation in order to count as "passing". This particular attempt doesn't appear to do that, at least not regularly.

Post reply on HN