Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI or Ain't: Eliza

zserge.com

Post reply on HN
Print view View original

51–60 of 112 posts

Re: AI or Ain't: Eliza

#51

Post by tempodox »

jll29 wrote: post #2

Here is the pointer to the original Eliza paper https://dl.acm.org/doi/10.1145/365153.365168 Note that Weizenbaum was an AI critic: Weizenbaum's intention was not for Eliza to pass the Turing test, but to show to people that a clearly not intelligent program based on primitive pattern matching can appear to behave intelligently. He failed: His own secretary wanted to be left alone with the software and typed in her p…

All this “AI” hype is a constant reminder to me that you cannot reason anybody out of something that they want to believe. People's need to believe in miracles is obviously stronger than all reason.

Re: AI or Ain't: Eliza

#52

Post by acidburnNSA »

NavinF wrote: post #49
acidburnNSA wrote: post #31

Earlier quoted context omitted.

Yes. I'm surprised to hear Eliza passed a turing test. SBAITSO was fun but pretty clearly not a human.

Eliza never passed a turing test. Nobody tried to test Eliza because they knew what the result would be

I'm responding to the quote in TFA:

> One of the first computer programs that successfully passed the Turing test was Eliza.

I haven't studied the history of it.

Re: AI or Ain't: Eliza

#54

Post by bandrami »

notahacker wrote: post #37
throwuxiytayq wrote: post #27

Earlier quoted context omitted.

The example ELIZA responses in the paper are so laughably bad and trivial to pick up, I'm not convinced the human interrogators were sober/conscious/awake during the experiment.

tbf the human side of those conversations isn't much better. I think if someone tried prompt injection hacks on me I'd be tempted to be politely obtuse to troll them right back. Turing's version involves experts who definitely aren't in the same room waving to each other, but the fundamental problem is it isn't a particularly good test

Is there a name for the reverse Turing test? How can a Python script convince me it's not actually a human?

Re: AI or Ain't: Eliza

#55

Post by pixl97 »

aatd86 wrote: post #47
stavros wrote: post #45

Earlier quoted context omitted.

But what does "act on its own" mean? If I give the program some randomness over its next action, is that "acting on its own"? When I'm at work, I act according to a series of instructions. Am I not acting on my own? This is a very philosophical discussion, but if I had an infinitely-powerful computer and could simulate an entire universe based on a series of instructions (physical laws), would the beings in that univ…

Yes, as long as the computer chooses its next set of instructions in order to maximize a given value (the objective), I would say that it acts on its own. Instruction set that was never defined by anyone that is. If the instruction set is limited and defined by someone else, I believe it doesn't. I think, re. the simulated universe, that for us, they wouldn't have free will because we know causality (as long as we ar…

>Partial information provides choice of action, therefore free will.

So how would an agent based system connected to a multi-modal LLM/AI fall into this?

Re: AI or Ain't: Eliza

#56

Post by the_af »

acidburnNSA wrote: post #52
NavinF wrote: post #49

Earlier quoted context omitted.

Eliza never passed a turing test. Nobody tried to test Eliza because they knew what the result would be

I'm responding to the quote in TFA: > One of the first computer programs that successfully passed the Turing test was Eliza. I haven't studied the history of it.

Mentioned in arstechnica to my surprise, but do note the paper wasn't peer reviewed and they mention flaws in the methodology.

I cannot believe anyone passingly familiar with ELIZA would be fooled by it.

Re: AI or Ain't: Eliza

#57

Post by pixl97 »

masswerk wrote: post #50
aldousd666 wrote: post #6

Eliza's meant to us to be an illustration of the problem. In good old fashioned AI sentiment, it illustrates the fact that you need another if statement for every new kind of construct you want to simulate. But you deign to simulate each thing, like say turning a verb into a gerund, by writing a specific "gerundification" routine. Another to swap the Mes to Yous, etc. this isn't how people think nor most modern AI. T…

With the small reservation that this is not how Eliza works. Eliza sits on top of MAD/SLIP which does all the heavy work and provides lists and integer indexes, which is what is processed by Eliza. This allows Eliza to work on decomposition rules, which isolate keywords per position and context, and transformation (composition) rules to recombine elements and links between those two. Meaning, the model is much more t…

>how eager we are to overestimate the capabilities of such a partner in conversation, as soon as some criteria seem to be satisfied. Which is arguably still relevant today.

Honestly, AI shouldn't be the takeaway point here, but how we do the same for politics.

Re: AI or Ain't: Eliza

#58

Post by pixl97 »

notahacker wrote: post #25
throwaway4aday wrote: post #15

> Interestingly, Eliza still outperforms ChatGPT in certain Turing test variations. I see we have a new entry for the 2024 Lies of Omission award. The article linked to plainly shows that Eliza only beats ChatpGPT-3.5 and is in the bottom half when ranked against a variety of different system prompts. An excellent ass covering strategy that relies on the reader not checking sources. An honest author would have actual…

"GPT-4 achieved a success rate of 41 percent, second only to actual humans" also feels like a (much bigger) lie of omission looking at the original paper. GPT4's performance was in the range of 6% to 41%, Eliza's 27% score sat in the upper middle of that range, and considering the bots tested consisted of 8 GPT4 prompts, 2 GPT3.5 prompts and a naive script from the 1960s, GPT4 would have had to be remarkably consiste…

Yea, It's really hard to get GPT to sound human because the RLHF really wants to let you know it's not a human.

GPT4 + a RLHF that was trained to think it was human would be a much different beast.

Re: AI or Ain't: Eliza

#59

Post by pixl97 »

__MatrixMan__ wrote: post #30
nbzso wrote: post #3

So the Turing test actually tests not a technology but the level of intelligence of the user. So, we are doomed, it seems.:)

He originally made the argument about gender, not intelligence. I think he was arguing for a whole class of properties for which there's no difference between authenticity and convincing fakery. I think the point is less that there is a truth and we're too dumb to figure it out, and more that in certain circumstances we'll just have to accept a lower bar for evidence about whether those properties apply. It reminds m…

Or another way to put this, it's not a binary problem, it's a probability continuum.

Even the definition of 'human intelligence' is a continuum from the smartest to the dumbest of us, that doesn't even stop there and descends thought all animal life.

Re: AI or Ain't: Eliza

#60

Post by derbOac »

nbzso wrote: post #3

So the Turing test actually tests not a technology but the level of intelligence of the user. So, we are doomed, it seems.:)

I've often felt that a better version is not whether a person can guess that it's AI or a human, but whether people behave and feel differently with an AI or human.

That's vague and covers a universe of criteria — mood, satisfaction with the conversation, actual behavior and so forth — but it also I think is a more realistic gauge of AI performance. It's probably unattainable but that's not necessarily a bad thing. If it is attainable within confidence then it's a pretty powerful AI.

There are probably some people who would be ok with some AI for some purposes.

Post reply on HN