Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

31–40 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#31

I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.

Might have completely missed something, but I thought AR/VR was (and still is) a solution looking for a problem. Have they finally stumbled into something people want to do with them, beyond games and porn?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#32
I'll never forget that day when I was a junior at University of Oregon, and had just finished reading GEB. Hofstadter was from UO, and I was taking a data structures class from Andrzej Proskurowski, who knew Hofstadter. Andrzej was a pretty brilliant man who had a reputation for being very blunt. I was in his office hours and I asked him for his take on the book. He said, "Hofstadter is first order...bullshitter."

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#33

" ...In my book," pretty much sums it up. Literally everyone who seems to pontificate on a Chat bot and new Photoshop features has a book and can't seem to help but mention it. Replace "book" with literally anything else and you can see it's completely about ego and money at the end of the day. He's probably getting ready to announce he accepted a position from one of these recently funded AI companies and whatever h…

Hofstadter clearly isn't trying to peddle his books for money or clout. He writes in depth, thoughtful books about the nature of consciousness, not guides to making money off of AI.

The book he mentions in this interview, I Am a Strange Loop, isn't some cash grab in response to LLMs - it was written in 2007.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#34
post #8

Hofstadter makes the claim that "these LLMs and other systems like them are all feed-forward". That doesn't sound right to me, but I'm only a casual observer of LLM tech. Is his assertion accurate? FWIW, ChatGPT doesn't think so. :-)

I think he was referring to feedforward when running GPT in a current conversation, it only remembers the conversation by re-running the prompts. It isn't doing 'feed-back' in the sense of re-updating its weights and learning while having the conversation. So during any one conversation it is only feed-forward.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#35

Do we really lack a good understanding of LLM's and deep nets that we need to be afraid of them? I would love to see this disproved with some open source work on the internals of these models and how they do inference and exactly reason. And why they could possibly never realize an AGI with the current stream of models. Being able to display human level intelligence and creative in confined spaces (be it Chess or Go…

I'm not afraid of LLMs, they are stupid machines. I'm afraid of what humans will use them for, with serious, irreversible real-life consequences.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#36

Do we really lack a good understanding of LLM's and deep nets that we need to be afraid of them? I would love to see this disproved with some open source work on the internals of these models and how they do inference and exactly reason. And why they could possibly never realize an AGI with the current stream of models. Being able to display human level intelligence and creative in confined spaces (be it Chess or Go…

No one has ever been convinced to fly an airplane into a building by a chess board. People have gotten convinced of that by words though. The “attack surface” on human sensibility is just enormous if you’re able to use believably human language.

Nobody has been convinced to fly an airplane into a building by a language model either.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#37
post #26

I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.

If it's a positive interaction, then what is your fear?

That it's a negative interaction pretending to be a positive one.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#38

In GEB Hofstadter dismisses the idea that AI could understand / compose / feel music like a human. I thought about this a lot when I started using GPT, especially early on when it demonstrated an ability to explain why things were funny or sad, intrinsically human qualities hitherto insulated from machine

Repeating the words thousands of humans have written about emotion doesn’t meant it feels them. A sociopath could define empathy and still be missing the deeper experience behind the emotion.

That can also be scary. I think you have a valid point, and that would mean the AI 'thinking' is more similar to a Sociopath. Can still be 'human' if just a broken human, or what some have also called 'alien'.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#39

Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes! >"I never imagined that computers would rival, let alone surpass, human intelligence. And in principle, I thought they could rival human intelligence. I didn't see any reason that they couldn't." Yeah, so he got fooled by LLMs and hasnt been burned by it failing to do the most basic logi…

People overly impressed by LLMs haven't spent a lot of time trying to make them actually useful. When you do, you learn that they're talented mimics but still quite limited.

I think the reason to be impressed is that they do things that were previously not possible. And they are absolutely directly useful! Just not for everything. But it seems like a very fruitful line of research, and it’s easy to believe that future iterations will have significant improvements and those improvements will happen quickly. There’s no sense worrying about whether GPT4 is smarter than a human, the interesting part is that it demonstrates that we have techniques that may be able to get you to a machine that is smarter than a human.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#40
post #8

Hofstadter makes the claim that "these LLMs and other systems like them are all feed-forward". That doesn't sound right to me, but I'm only a casual observer of LLM tech. Is his assertion accurate? FWIW, ChatGPT doesn't think so. :-)

I believe what he is referring to is that the LLM’s weights are set when chatting. It is not “learning”. simply using its pretrained weights on your input.

Edit: Nope. TIL feed-forward means no loops.

Post reply on HN