Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

21–30 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#21

Post by climatologist »

jurgenaut23 wrote: post #9

I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…

I think they are not good at expressing what exactly they fear. Both understand that people are embedded in culture and tools which act as cognitive aids and guides. Their fear is that people will start trusting these artificial systems too much and delegating too many important decisions to systems that have no understanding of what it means to be human. This is already happening with children that grew up with iPhones and ubiquitous internet connectivity. Their mental habits are markedly different from the people that did not have these affordances.

I don't think this is an existential threat but I also understand why they are afraid because I've seen what happens to kids who have their phones taken away.

These systems are extremely brittle and prone to all sorts of weird failures so as people start relying on them more and more the probability of catastrophic failures also starts to creep up. All it takes is a single grid failure to show how brittle the whole thing really is and I think that's what they're failing to properly express.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#22

Post by taeric »

hospitalJail wrote: post #3

Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes! >"I never imagined that computers would rival, let alone surpass, human intelligence. And in principle, I thought they could rival human intelligence. I didn't see any reason that they couldn't." Yeah, so he got fooled by LLMs and hasnt been burned by it failing to do the most basic logi…

I mean, this someone "who is past their prime" is a very respected someone that almost certainly inspired a fair number of the folks working in these fields.

So, yes, this is largely mining for quotes. But those are great quotes to ponder and to echo back through the machines that are research and progress to see where they lead.

It would be one thing if these were being taken as a "stop all current work to make sure we are aligned with those that came before us." I don't see it in that way, though. Rather, there is value in listening to those that went before.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#23

Post by fipar »

hospitalJail wrote: post #3

Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes! >"I never imagined that computers would rival, let alone surpass, human intelligence. And in principle, I thought they could rival human intelligence. I didn't see any reason that they couldn't." Yeah, so he got fooled by LLMs and hasnt been burned by it failing to do the most basic logi…

> Its a language model and it fooled DH. It hasnt gotten smarter than us yet. Its just faster at repeating what other humans verbally said.

I agree with this completely. That said, I think this part is a bit unfair:

> Lets take someone, who is past their prime and interview them on a topic they have never worked on.

AI has been a part of DH's work for decades. For most of that time, he's dismissed the mainstream approach as being intelligent in the Strange Loop sense, and was involved in alternative approaches to AI.

Also, if we remove "faster", "repeating what other humans verbally said" is something a lot of humans do, especially little children. I think that may be the part that scares DH: at this point, what these models are doing is not that different (from a superficial POV) from what little kids or even some adults do.

I still agree with you though, and I think what DH misses here is the fact that IMHO there is no introspection at all in these models. Somehow, in the case of humans, we go from parroting like an LLM to introspection and actually original productions over the course of some years (n = 2 for me, but I think any parent can confirm this; watching this happen in front of my eyes has been one of greatest experiences in my life), but I can still understand how someone like DH would be confused by the current state of affairs.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#24

Post by mistermann »

EA-3167 wrote: post #17
ethanbond wrote: post #10

Earlier quoted context omitted.

No one has ever been convinced to fly an airplane into a building by a chess board. People have gotten convinced of that by words though. The “attack surface” on human sensibility is just enormous if you’re able to use believably human language.

Well... words and decades of circumstances. If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. I think we tend to credit words where often circumstances are doing the heavy lifting. For example try to start a riot with words in Rodeo Drive. Now try to do it in Nanterre. Or better yet, try to start a riot in Nanterre before a 17 year was…

> Well... words and decades of circumstances.

That also can be modified with words though (but for both good and bad). Unfortunately, those with expertise in this domain may not have all of our best interests at heart.

> If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow.

There's also the problem of non-religious faith based belief.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#25

Post by iamleppert »

" ...In my book," pretty much sums it up. Literally everyone who seems to pontificate on a Chat bot and new Photoshop features has a book and can't seem to help but mention it. Replace "book" with literally anything else and you can see it's completely about ego and money at the end of the day. He's probably getting ready to announce he accepted a position from one of these recently funded AI companies and whatever he's getting from that is more than he could make from his book, academia and giving interviews where he talks about humanity's fear of fire.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#26

Post by Kiro »

detourdog wrote: post #15

I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.

If it's a positive interaction, then what is your fear?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#27

Post by frankfrank13 »

iamleppert wrote: post #25

" ...In my book," pretty much sums it up. Literally everyone who seems to pontificate on a Chat bot and new Photoshop features has a book and can't seem to help but mention it. Replace "book" with literally anything else and you can see it's completely about ego and money at the end of the day. He's probably getting ready to announce he accepted a position from one of these recently funded AI companies and whatever h…

idk about that, this isn't some rando twitter bandwagoner. This is a legendary thinker and writer

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#28

Post by FrustratedMonky »

iamleppert wrote: post #25

" ...In my book," pretty much sums it up. Literally everyone who seems to pontificate on a Chat bot and new Photoshop features has a book and can't seem to help but mention it. Replace "book" with literally anything else and you can see it's completely about ego and money at the end of the day. He's probably getting ready to announce he accepted a position from one of these recently funded AI companies and whatever h…

I initially thought so too. But his wording wasn't very self-building, it was all about how he was wrong, scared, etc... Overall not the take (wording) I'd expect from someone doing self promotion.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#29

Post by frankfrank13 »

jurgenaut23 wrote: post #9

I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…

Agreed 100%. He's brilliant, but he's not a programmer. IMO he's been tricked, in a way that Chompsky was not.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#30

Post by chairhairair »

EA-3167 wrote: post #17
ethanbond wrote: post #10

Earlier quoted context omitted.

No one has ever been convinced to fly an airplane into a building by a chess board. People have gotten convinced of that by words though. The “attack surface” on human sensibility is just enormous if you’re able to use believably human language.

Well... words and decades of circumstances. If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. I think we tend to credit words where often circumstances are doing the heavy lifting. For example try to start a riot with words in Rodeo Drive. Now try to do it in Nanterre. Or better yet, try to start a riot in Nanterre before a 17 year was…

Many people are in desperate situations. Most of them are not convinced to do much about it.

Words can aim discontent.

Were the economic conditions in Weimar Germany that much worse than many places today?

Post reply on HN