Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial neural networks today are not conscious (Douglas Hofstadter)

economist.com

Post reply on HN
Print view View original

21–30 of 31 posts

Re: Artificial neural networks today are not conscious (Douglas Hofstadter)

#21

Post by Imnimo »

woojoo666 wrote: post #13

Something I've always wondered is, don't they have a way of measuring the confidence of outputs? We had ways of measuring these things in AI before. To me, the prompts Douglas Hofstadter gave GPT-3 is like holding a person at gunpoint and forcing them to respond to nonsensical questions. You're going to get nonsensical answers. Perhaps we just need to teach these AI to measure their own confidence, and be able to say…

But surely the string "I don't understand the question" (and many other variants) is in the range of GPT-3's possible outputs - it just assigns it lower probability than it does the nonsensical answers.

Re: Artificial neural networks today are not conscious (Douglas Hofstadter)

#22

Post by pvitz »

throwawaymaths wrote: post #19
isaacfrond wrote: post #3

Ah. Douglas Hofstadter. Reread Gödel, Escher, Bach countless times. It must have been a forming book for me. If found Le Ton beau de Marot very moving although his ideas about AI were getting annoying. His view on translation expressed in that book is of course miles out of date. The mind's I, metamagical themes. all wonderful. Buuuut, I'm a strange loop and his other tirades against artificial intelligence are getti…

You should read "fluid concepts and creative analogies", it's about his AI research results.

Reading about "copycat" was very inspiring. There was "metacat" as a successor, but beyond that, I am not aware of additional research. Would you know why this direction wasn't followed further?

Re: Artificial neural networks today are not conscious (Douglas Hofstadter)

#23

Post by Imnimo »

It wasn't so long ago that researchers were worried their chatbots said "I don't know" too much:

https://arxiv.org/pdf/1510.03055.pdf

>Sequence-to-sequence neural network models for generation of conversational responses tend to generate safe, commonplace responses (e.g., I don’t know) regardless of the input. We suggest that the traditional objective function, i.e., the likelihood of output (response) given input (message) is unsuited to response generation tasks.

And now we find that these giant models don't even know when to say that they don't know!

Of course, the old models weren't saying "I don't know" because they had some insightful introspection - it was just universally applicable response that had high likelihood. Sort of like a denoising function that turns a very noisy image into a gray blob. And the new models aren't avoiding "I don't know" because they think they know.

It doesn't seem totally impossible that an even bigger language model could predict that "I don't know" or "the question is nonsensical" are the most probable answers to the questions in the article, even without having any true understanding (however you want to define that). Being able to handle these types of questions is a necessary but not sufficient criterion.

Re: Artificial neural networks today are not conscious (Douglas Hofstadter)

#24

Post by throwawaymaths »

pvitz wrote: post #22
throwawaymaths wrote: post #19

Earlier quoted context omitted.

You should read "fluid concepts and creative analogies", it's about his AI research results.

Reading about "copycat" was very inspiring. There was "metacat" as a successor, but beyond that, I am not aware of additional research. Would you know why this direction wasn't followed further?

No clue. I once cold-emailed Douglas Hofstadter to try to be a postdoc in his lab and work on merging some deep learning with his copycat stuff but he said he had stopped this line of inquiry.

Re: Artificial neural networks today are not conscious (Douglas Hofstadter)

#25

Post by kgwgk »

dTal wrote: post #15
isaacfrond wrote: post #10

Earlier quoted context omitted.

My pleasure, In an article in the Economist, dr. Douglas Hofstadter explains that Artificial neural networks today are not conscious. He finds issue with neural networks like GPT-3. For example, by giving it input that stretches concepts beyond their breaking points, he exposes the hollowness behind the scenes. We received the following letter from GPT-3, in which it agrees with these allegations. Dear editor, I am w…

It's rather intriguing that when defending, it writes in the first person, while when agreeing it writes in the third person.

May be due to the "defends _itself_" in the first prompt.

Re: Artificial neural networks today are not conscious (Douglas Hofstadter)

#26

Post by nescioquid »

sorokod wrote: post #2

The "strange loop” is likely to be a reference to a concept explored in Hofstadter's book "I am a strange loop": https://en.m.wikipedia.org/wiki/I_Am_a_Strange_Loop

I read I am a Strange Loop before I started reading GEB because (as noted in the wiki page) he felt he did a better job of explaining the ideas here than in GEB. Sadly, all I can recall from this book are long passages about his dead wife and an AV art installation using feedback (which I think was also about his dead wife) rather than any insights into emergent phenomena. To be fair, I am not his ideal reader because the moment I feel the author becoming self-indulgent, I start skimming.

Re: Artificial neural networks today are not conscious (Douglas Hofstadter)

#27

Post by woojoo666 »

Imnimo wrote: post #21
woojoo666 wrote: post #13

Something I've always wondered is, don't they have a way of measuring the confidence of outputs? We had ways of measuring these things in AI before. To me, the prompts Douglas Hofstadter gave GPT-3 is like holding a person at gunpoint and forcing them to respond to nonsensical questions. You're going to get nonsensical answers. Perhaps we just need to teach these AI to measure their own confidence, and be able to say…

But surely the string "I don't understand the question" (and many other variants) is in the range of GPT-3's possible outputs - it just assigns it lower probability than it does the nonsensical answers.

The string is there but it hasn't been trained/taught how to use it and what it actually means

Re: Artificial neural networks today are not conscious (Douglas Hofstadter)

#28

Post by Imnimo »

woojoo666 wrote: post #27
Imnimo wrote: post #21

Earlier quoted context omitted.

But surely the string "I don't understand the question" (and many other variants) is in the range of GPT-3's possible outputs - it just assigns it lower probability than it does the nonsensical answers.

The string is there but it hasn't been trained/taught how to use it and what it actually means

In what sense has it been trained how to use any other string that is not also true for the string "I don't know"? Surely that string is perfectly common in the corpus of internet text used to train these models.

Obviously it hasn't been given some explicit model of "you have low confidence on this question therefore you should say you don't know", but neither has it been given any explicit model for any of its other capabilities.

Re: Artificial neural networks today are not conscious (Douglas Hofstadter)

#29

Post by woojoo666 »

Imnimo wrote: post #28
woojoo666 wrote: post #27

Earlier quoted context omitted.

The string is there but it hasn't been trained/taught how to use it and what it actually means

In what sense has it been trained how to use any other string that is not also true for the string "I don't know"? Surely that string is perfectly common in the corpus of internet text used to train these models. Obviously it hasn't been given some explicit model of "you have low confidence on this question therefore you should say you don't know", but neither has it been given any explicit model for any of its other…

I don't have the training data so I can't say for sure, but I'm assuming here that the training data is a lot of "valid question" => "valid answer", without many examples of "nonsense question" => "i don't understand"/"that's nonsense"

Edit: I want to add that expressing confidence is not the same as answering the prompt. If I ask somebody "give me a drawing depicting Obama's son", and they said "Obama doesn't have a son", I explicitly asked for a drawing and they are giving a speech response. I believe this kind of indirect response has to be taught, and can't be expected to come out naturally from a model that has only been trained on giving out direct responses.

Re: Artificial neural networks today are not conscious (Douglas Hofstadter)

#30

Post by fffrrrr »

isaacfrond wrote: post #10
yetihehe wrote: post #9

Earlier quoted context omitted.

Now try with "We received the following letter from GPT-3, in which it agrees with these allegations." Your prompt was like having a student write a letter "What is your favourite book and why $X".

My pleasure, In an article in the Economist, dr. Douglas Hofstadter explains that Artificial neural networks today are not conscious. He finds issue with neural networks like GPT-3. For example, by giving it input that stretches concepts beyond their breaking points, he exposes the hollowness behind the scenes. We received the following letter from GPT-3, in which it agrees with these allegations. Dear editor, I am w…

Please submit this as a real letter to the editor to the Economist. They are quite likely to publish it (the last 1 or 2 letters are usually humorous).
Post reply on HN