Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI or Ain't: Eliza

zserge.com

Post reply on HN
Print view View original

21–30 of 112 posts

Re: AI or Ain't: Eliza

#21

Post by stavros »

adestefan wrote: post #13
jll29 wrote: post #2

Here is the pointer to the original Eliza paper https://dl.acm.org/doi/10.1145/365153.365168 Note that Weizenbaum was an AI critic: Weizenbaum's intention was not for Eliza to pass the Turing test, but to show to people that a clearly not intelligent program based on primitive pattern matching can appear to behave intelligently. He failed: His own secretary wanted to be left alone with the software and typed in her p…

Weizenbaum even wrote an entire book, Computer Power and Human Reason: From Judgment To Calculation, on how AI is a crank term for computer science. The basic premise is that humans are not machine so stop using language that treats them as such. Especially that computers can decide what comes next, but only a human can choose what to do. The book also has one of the best and most succinct descriptions of Turing mach…

> Especially that computers can decide what comes next, but only a human can choose what to do.

I don't understand this, all the programs I've ever written make decisions based on some factors.

Are you talking about free will? If so, what is free will?

Re: AI or Ain't: Eliza

#22

Post by throwaway4aday »

masswerk wrote: post #20
throwaway4aday wrote: post #15

> Interestingly, Eliza still outperforms ChatGPT in certain Turing test variations. I see we have a new entry for the 2024 Lies of Omission award. The article linked to plainly shows that Eliza only beats ChatpGPT-3.5 and is in the bottom half when ranked against a variety of different system prompts. An excellent ass covering strategy that relies on the reader not checking sources. An honest author would have actual…

Hum, note that this was not an argument about or against GPT, but about the "unreasonable" success of a, by all standards, primitive algorithm that manages to get (somewhat) away by crafting the pre- and context of the conversation. By no means, on the other hand, could I read this article and understand it as claiming any superiority over modern applications. (Nobody with even the crudest understanding of the princi…

Several people in this thread appear to have misunderstood due to the way this article was written.

Re: AI or Ain't: Eliza

#25

Post by notahacker »

throwaway4aday wrote: post #15

> Interestingly, Eliza still outperforms ChatGPT in certain Turing test variations. I see we have a new entry for the 2024 Lies of Omission award. The article linked to plainly shows that Eliza only beats ChatpGPT-3.5 and is in the bottom half when ranked against a variety of different system prompts. An excellent ass covering strategy that relies on the reader not checking sources. An honest author would have actual…

"GPT-4 achieved a success rate of 41 percent, second only to actual humans" also feels like a (much bigger) lie of omission looking at the original paper. GPT4's performance was in the range of 6% to 41%, Eliza's 27% score sat in the upper middle of that range, and considering the bots tested consisted of 8 GPT4 prompts, 2 GPT3.5 prompts and a naive script from the 1960s, GPT4 would have had to be remarkably consistently inhuman not to finish "second only to humans" with its highest scoring prompt

The blog appears to have been updated to specify GPT3.5, but the original version was accurate.

The paper itself is interesting as it covers the limitations (it has big methodological issues), how the GPT prompts attempted to overridei default chatGPT tone and reasons why ELIZA performed surprisingly well (some thought it was so uncooperative, it must be human!) https://arxiv.org/pdf/2310.20216.pdf

Re: AI or Ain't: Eliza

#26

Post by aatd86 »

stavros wrote: post #21
adestefan wrote: post #13

Earlier quoted context omitted.

Weizenbaum even wrote an entire book, Computer Power and Human Reason: From Judgment To Calculation, on how AI is a crank term for computer science. The basic premise is that humans are not machine so stop using language that treats them as such. Especially that computers can decide what comes next, but only a human can choose what to do. The book also has one of the best and most succinct descriptions of Turing mach…

> Especially that computers can decide what comes next, but only a human can choose what to do. I don't understand this, all the programs I've ever written make decisions based on some factors. Are you talking about free will? If so, what is free will?

Doing things because you can and not because you have to? Creative endeavors in the largest sense?

Re: AI or Ain't: Eliza

#27

Post by throwuxiytayq »

notahacker wrote: post #25
throwaway4aday wrote: post #15

> Interestingly, Eliza still outperforms ChatGPT in certain Turing test variations. I see we have a new entry for the 2024 Lies of Omission award. The article linked to plainly shows that Eliza only beats ChatpGPT-3.5 and is in the bottom half when ranked against a variety of different system prompts. An excellent ass covering strategy that relies on the reader not checking sources. An honest author would have actual…

"GPT-4 achieved a success rate of 41 percent, second only to actual humans" also feels like a (much bigger) lie of omission looking at the original paper. GPT4's performance was in the range of 6% to 41%, Eliza's 27% score sat in the upper middle of that range, and considering the bots tested consisted of 8 GPT4 prompts, 2 GPT3.5 prompts and a naive script from the 1960s, GPT4 would have had to be remarkably consiste…

The example ELIZA responses in the paper are so laughably bad and trivial to pick up, I'm not convinced the human interrogators were sober/conscious/awake during the experiment.

Re: AI or Ain't: Eliza

#28

Post by stavros »

aatd86 wrote: post #26
stavros wrote: post #21

Earlier quoted context omitted.

> Especially that computers can decide what comes next, but only a human can choose what to do. I don't understand this, all the programs I've ever written make decisions based on some factors. Are you talking about free will? If so, what is free will?

Doing things because you can and not because you have to? Creative endeavors in the largest sense?

Does a program have to do things? What can it do? What does a human have to/can do?

Re: AI or Ain't: Eliza

#29

Post by the_af »

acidburnNSA wrote: post #24

So was Eliza from 1966 more intelligent than Dr SBAITSO from 1990? https://en.wikipedia.org/wiki/Dr._Sbaitso

Wasn't SBAITSO pretty much the same idea as Eliza, only using speech synthesis with a Soundblaster?

Re: AI or Ain't: Eliza

#30

Post by __MatrixMan__ »

nbzso wrote: post #3

So the Turing test actually tests not a technology but the level of intelligence of the user. So, we are doomed, it seems.:)

He originally made the argument about gender, not intelligence. I think he was arguing for a whole class of properties for which there's no difference between authenticity and convincing fakery.

I think the point is less that there is a truth and we're too dumb to figure it out, and more that in certain circumstances we'll just have to accept a lower bar for evidence about whether those properties apply.

It reminds me of how no class of computer can solve the halting problem for itself. No matter how intelligent you are, there will be holes like this in your certainty about some things.

Post reply on HN