Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

261–270 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#261

Post by 13years »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Those with great visibility often are not the greatest minds. However, in a broader sense we would have been served well if philosophizers would have had more input on the development of social media tech. It is an example of engineers knowing a field narrowly and deeply but not understanding the societal consequence.

AI may very well fall into the same pattern and it is something I have written out in some detail of thoughts around alignment and the traps of both ego and misunderstanding human nature for which we want to model alignment.

https://www.mindprison.cc/p/ai-singularity-the-hubris-trap

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#262

Post by ke88y »

sanp wrote: post #259
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

The warnings are an admission from the “Great Minds and a great Thinkers” that someone other than them has created something previously thought impossible or, at the very least, several years / decades out. So, I am not sure ego is at play here. Perhaps someone close to the problem (your actual scientists on the ground) is not capable / unwilling to accept the issues raised as that has direct (negative) impact on wha…

Perhaps. It's also possible that they don't understand how dangerous what they've created is because it feels so "normal" and "pedestrian" to them. See also: nuclear scientists getting radiation poisoning, I suppose. But that's also true for all of the other people being discussed, I would think.

But I do think "wants to be in the discourse and on top" is a pretty strong correlate with the degree to which someone characterizes these as "concerns" vs "x-risk".

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#263

Post by leftcenterright »

tasuki wrote: post #237
lo_zamoyski wrote: post #206

Earlier quoted context omitted.

> The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. Except they're not philosophizing, not in any real sense of the word. They're terrible at it. Most of them are frauds, quacks, and pseudo-intellectual con artists (like Harari) who adore the limelight offered to them by the media and a TED Talks-watching segment of t…

> frauds, quacks, and pseudo-intellectual con artists (like Harari) Uh, mind elaborating? Why is Harari that? Do you have any examples of non-frauds and actual intellectuals? > a TED Talks-watching segment of the public who are, frankly, intellectually out of their depth, but enjoy the delusion and feeling of participating in something they think is "intellectual" I'm afraid that would be me.

> I'm afraid that would be me.

I think a lot of us are that, personally speaking: honest sincere analyses and investing time into critical analysis almost always will bring out more than what we hear in a talk. It does take a big amount of effort though than just watching a talk while munching on some snack.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#264

Post by lowbloodsugar »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

In the "scientists complacent about dangers of the systems they are working on" file, I think we are all familiar with https://en.wikipedia.org/wiki/Demon_core.

Now, admittedly, this was scientists being complacent about a thing they knew was dangerous, whereas here we are saying scientists don't think their thing is dangerous. But very clearly, AI could be dangerous, so it's more that these scientists don't think their system could be dangerous. Presumably the scientists and engineers behind the https://en.wikipedia.org/wiki/Therac-25 didn't think it would kill people.

So maybe the problem is precisely that when we bring up extinction events from AGI, scientists rolling their eyes is the very reason we should be fucking worried. Their contempt for the possibility of the threat is what will get us killed.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#265

Post by jimbokun »

ralphc wrote: post #107

When I read about the dangers of AI I'm reminded of the feeling I had after reading Jeff Hawkin's "On Intelligence". He talked about simulating the neocortex to do many of the things that deep learning and LLM's are doing now. His research may or may not be a dead end but his work, and this work, to me seems like we're building the neocortex layer without building the underlying "lizard brain" that higher animals' br…

Or until someone decides to give AI a "lizard brain" and sets it loose.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#266

Post by pbw »

arketyp wrote: post #243
pbw wrote: post #198

Earlier quoted context omitted.

My point is you cannot teach a cockroach calculus, but if AIs invent a new type of math, they would be able to teach it to us. That's my claim. So the analogy of "we are cockroaches compared to the AI" is wrong, that won't be the case. Once you have "enough" intelligence to have a complex language, like we do, I'm claiming you are in the club of intelligent species, and all species in that club can communicate ideas…

I'm reminded of how one writes programs. I cannot maintain the state of the machine in my head, but I can convince myself of its workings, its intelligence, by reading the code, following along on its line of reasoning as it were. I think the Intelligence Completeness may boil down to the very same Church-Turing thesis.

Yes I agree it might be same thing under the hood. But with intelligence many very smart people seem to fall into using these analogies that diminish humans in a way I don't think is accurate. And I feel that makes people more scared of AI than they need to be, makes AI seem totally alien. [1]

The AIs might spit out entire fields of knowledge, and it might take humans decades of study to understand it all. And no single human might actually understand it all at the same time. But that's how very advanced fields of study already are.

But the "cockroach" slur implies AIs would be in this other stratosphere having endless discussions that we cannot remotely grok. My guess is that won't happen. Because if the AI were to say "I cannot explain this to you" I'd take that as evidence it wasn't all that intelligent after all.

[1] - https://metastable.org/alien.html

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#267

Post by DennisP »

lolinder wrote: post #203
ctoth wrote: post #163

Earlier quoted context omitted.

A few years back (let's call it 2020) and autonomous cars, which are being used for taxi trips today, would be five years in the future. In fact they would be three. Unless something major happens in the next two years, there will still be self-driving cars, even more of them, driving and picking up people in 2025. This is not the argument you think it is.

Self-driving cars currently operate in extremely controlled conditions in a few specific locations. There's very little evidence that they're on a trajectory to break free of those restrictions. It doesn't matter how much an airliner climbs in altitude, it's not going to reach LEO. Self-driving cars will not revolutionize the roads on the timescale that people thought it would, but the effort we put into them brought…

Tesla FSD isn't restricted to specific locations, and seems to be reducing the number of human interventions per hour at a pretty decent pace.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#268

Post by jimbokun »

pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

Isn't the difference between us and cockroaches just "we can think much quicker, see patterns and make insights better and faster than cockroaches"?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#269

Post by flangola7 »

JimtheCoder wrote: post #227
tyre wrote: post #164

Earlier quoted context omitted.

This is a great way to put it. The counter to "current generation AI is terrifying" seems to fall along the lines of it not being nearly as close to AGI as the layperson believes. But I don't think that matters. I don't believe that LLMs or image/voice/video generative models need to do much beyond what they can do today in order to wreck civilization level disaster. They don't need to become Skynet, learn to operate…

Doesn't this only work if everyone is stupid and believes everything they see/read/hear on the internet? You have to have a little more faith in humanity than that...

With respect: *Have you been living under a rock?*

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#270

Post by nielsbot »

hackeraccount wrote: post #16
nielsbot wrote: post #7

Earlier quoted context omitted.

Part of me wonders, though, could we "just" connect up an inference engine and voila? We could really be on a cusp of general AI. (Or it could be a ways off) That's a bit frightening in several ways.

I kind of expected AI to be AI - and not a mirror.

Meaning it wouldn't necessarily be human-like?
Post reply on HN