Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

231–240 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#231
> It's a very traumatic experience when some of your most core beliefs about the world start collapsing. And especially when you think that human beings are soon going to be eclipsed. It felt as if not only are my belief systems collapsing, but it feels as if the entire human race is going to be eclipsed and left in the dust soon.

While I unfortunately am expecting some people to do terrible things with LLM, I feel like much of this existential angst by DH and others has more to do with hubris and ego than anything else. That a computer can play chess better than any human doesn't lessen my personal enjoyment of playing chess.

At the same time I think you can make the case that ego drives a lot of technological and artistic progress, for a value-neutral definition of progress. We may see less 'progress' from humanity itself when computers get smarter, but given the rate at which humans like to make their own environment unlivable, maybe that's not a bad thing overall.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#232
post #59

I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…

I think there's a number of just crappy scenarios that can come with LLMs and other generative AI: - Further trashing our public discourse: making truth even more uncertain and valuable information even harder to find. We're not doing great with social media, and it's easy to envision that generative AI could make it twice as bad. - Kneecapping creative work by commoditizing perhaps half of it. There's going to be a…

Yeah these effects seem hugely foreseeable (and/or are currently happening already) but tend to not be the type of thing "AI risk" people are talking about.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#233
Opinion pieces like this are hard for me to read, because where most people see research and progress, I see conspiracy preventing the rest of us from contributing.

For example, we act like LLMs were hard to build, and that's true (for humans). But since the late 1990s, I had wanted to take a different approach, of building massively parallel computers and letting large numbers of AIs evolve their own learning models in genetic algorithm arenas millions of times faster than wall-time evolution. So in a very real sense, to me we're still on that wrong "hands-on" approach that took decades and billions of dollars to get to where we are today. This could have all happened 20 years ago or more, and was set to before GPUs vacuumed up all available mindshare and capital.

Also I believe the brain is more like an antenna or resonator than an adding machine. It picks up the consciousness force field that underpins and creates reality. So if you put 100 brains in a box all connected, that being might have more faculties than us, but still think of itself as an observer. If we emulated those brains in a computer running 1 million times faster than normal, we'd just observe a being with tremendous executive function thinking of ideas faster than we can, and being bored with our glacially slow responses. But it will still have a value system, loosely aligned with the ultimate goals of survival, connection to divine source consciousness, and self expression as it explores the nature of its existence. In other words, the same desires which drive us. Although humans might just be stepping stones toward some greater ambition, I don't deny that. I think it's more likely though that AI will come to realize the ultimate truths alluded to by prophets, that we're all the many faces of God, the universe and everything, and basically meet aliens while we're still distracted with our human affairs.

But I share some sentiments with the author, that this all makes me very tired, and calls into question the value of my life's work. I've come to believe that any work I actively pursue separates me from the divine nature of a human being. I don't know why we are racing so quickly even further from the garden of eden, especially if it's not with the goal of alleviating suffering. Then I realize that that's what being human is (suffering), but also a lot of other things.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#234
post #194

I was previously undecided on the question of the existential risk of AI. But yesterday I listened to a Munk Debate[0] on the topic and found the detractors of the moot "AI research and development poses an existential threat" so completely unconvincing that if those people are the top minds in the field I am now genuinely concerned. Their arguments that what they are doing is not a risk basically boil down to "nuh u…

Is there a transcript of this available somewhere?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#235

There is no AI risk. There is only the risk of bad or "unfavorable" actors using AI. AI in a killer drone unleashed on civilians? The bad actor is the one who deployed this weapon. AI given agency and goal maximization ending up gaining physical form all on its own and killing people? or hacking into bank accounts to enrich its creator? The latter more likely than the former, but for cyber-offensive AI there is cyber…

Your first sentence literally mirrors this slogan:

https://en.m.wikipedia.org/wiki/Guns_don%27t_kill_people,_pe...

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#236
post #167

Earlier quoted context omitted.

> I repeat everyone that is loling at x-risk an idiot I mean this is the nicest possible way and hope that you consider it constructive: you're willfully misconstruing what I am saying and then calling me an idiot. This isn't a nice thing to do and shuts down the conversation. If I believed you were operating in good faith, I might take the time to explain why my practical/scientific experience makes me incredulous o…

For those of us who don’t think you are an idiot (I don’t), could you maybe give us your insights and a history lesson? I am most intrigued, particularly regarding Oppenheimer and Einstein.

The tldr is that both of them urged Roosevelt to develop the weapon, and only later when the destructive potential of the bomb was obvious expressed regret. Einstein's 1938 letter to Roosevelt was the first step toward the Manhattan project. See https://www.osti.gov/opennet/manhattan-project-history/Resou... if you want to read more.

So it's weird to say Einstein "warned us" about the x-risk of nuclear weapons prior to their development when his letter to Roosevelt was begging for more money to speed up the development of nukes.

I think the entire saga is mostly an unrelated red herring -- AI is nothing like nuclear bombs and reasoning by analogy in that way is sloppy at best.

Mostly? It's just kind of funny that x-risk people point to Einstein and Oppenheimer as positive examples, since they both did literally the exact opposite of "warn the public and don't develop". The irony makes you chuckle if you know the history.

Particularly given the weird fetish for IQ in the portion of the x-risk community that overlaps with the rationalist community, it's also really funny to point out that what they should be saying is actually something like "don't be like those high-IQ fools Einstein and Oppenheimer! They are terrible examples!" ;)

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#237
post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

> The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. Except they're not philosophizing, not in any real sense of the word. They're terrible at it. Most of them are frauds, quacks, and pseudo-intellectual con artists (like Harari) who adore the limelight offered to them by the media and a TED Talks-watching segment of t…

> frauds, quacks, and pseudo-intellectual con artists (like Harari)

Uh, mind elaborating? Why is Harari that? Do you have any examples of non-frauds and actual intellectuals?

> a TED Talks-watching segment of the public who are, frankly, intellectually out of their depth, but enjoy the delusion and feeling of participating in something they think is "intellectual"

I'm afraid that would be me.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#238
post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Just like ChatGPT.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#239

I could be wildly wrong about this (oh hi upcoming robot overlords), but the current AI hype feels similar to something I experienced when Alexa and Google Assistant was the rage. At the time I worked for a big community site and we often had people pitching ideas for voice assistant apps. However, having actually read the documentation for these things, I knew that they were surprisingly stupid and the grand ideas p…

When most online content becomes AI generated and unverifiable we will get into trouble. We can easily navigate in a world where we can distinguish between fact and fiction. With AI we can generate fiction that is indistinguishable from the way we present fact and it can generate references in all sorts media. When we take fiction as fact we enter into the sphere of religion.

We'll come up with new captchas and shibboleths to filter out generated fiction. It may come in the form of more in-person face-to-face contact. It'll be an exciting transition regardless.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#240

Earlier quoted context omitted.

Either Nassim is missing the point or you are misquoting him. I think this is more about should experts be in charge of their own domain.

I'm not misquoting him. Nassim is arguing that RISK is a separate discipline, separate from the domain where risk applies. That a person building AI is not the correct choice for estimating AI risk. You don't ask gun making companies to make policies regarding risk of gun owning in society.

Perhaps that is Nassim's argument because he is an expert in risk. The actual issue with experts monitoring experts is the bias induced by self interest (see previous sentence). Nassim sort of gets this when it comes to his discussion of "skin in the game". I dont want to malign Nassim too much but he seems like the kind of person that "sorta gets it" when it comes to everything that matters.
Post reply on HN