Douglas Hofstadter changes his mind on Deep Learning and AI risk • Hacker News Board

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#151

Post by jrflowers » Mon, Jul 03, 2023, 6:29 PM UTC

hackinthebochs wrote: post #139

jrflowers wrote: post #114

Earlier quoted context omitted.

It’s interesting how confidently and obtusely people will proclaim categorical knowledge of the future. It is a little disconcerting that there is a fight between two somewhat cultish sects when it comes to language models. Both sides call them “artificial intelligence”, one side says they’ll save the world, the other side says they’ll end it. There is very little room to even question “Is this actually AI that we’re…

Trying to impute the motives of ones interlocutor is dumb and boring. How about we discuss the first-order issue instead. Here's my argument for why x-risk is a real possibility: The issue is that small misalignments in objectives can have outsized real-world effects. Optimizers are constrained by rules and computational resources. General intelligence allows an optimizer to find efficient solutions to computational…

> Trying to impute the motives of ones interlocutor is dumb and boring.

I know right? You should see the response to my point that nobody has been convinced to fly a plane into a building by an LLM. “Dumb and boring” hits the nail on the head.

> Seeing how confidently and obtusely people dismiss the risks of AI

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#152

Post by yanderekko » Mon, Jul 03, 2023, 6:30 PM UTC

mostlysimilar wrote: post #6

jackconsidine wrote: post #2

In GEB Hofstadter dismisses the idea that AI could understand / compose / feel music like a human. I thought about this a lot when I started using GPT, especially early on when it demonstrated an ability to explain why things were funny or sad, intrinsically human qualities hitherto insulated from machine

Repeating the words thousands of humans have written about emotion doesn’t meant it feels them. A sociopath could define empathy and still be missing the deeper experience behind the emotion.

Someday, when the AI releases a nanobot swarm comes to kill us all, a philosopher's last words will be "yes, but is it truly intelligent?" before he is broken down into biodiesel that will be used to power the paperclip factories.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#153

Post by arisAlexis » Mon, Jul 03, 2023, 6:31 PM UTC

ralphc wrote: post #107

When I read about the dangers of AI I'm reminded of the feeling I had after reading Jeff Hawkin's "On Intelligence". He talked about simulating the neocortex to do many of the things that deep learning and LLM's are doing now. His research may or may not be a dead end but his work, and this work, to me seems like we're building the neocortex layer without building the underlying "lizard brain" that higher animals' br…

That doesn't change the risk profile one bit though.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#154

Post by api » Mon, Jul 03, 2023, 6:31 PM UTC

echelon wrote: post #106

zone411 wrote: post #102

Earlier quoted context omitted.

That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...

If these people were so concerned, they'd by shouting from the hilltops and throwing their entire life savings into stopping us. They would organize workplace walkouts and strikes. There would be protests and banners. Burning data centers. Eliezer is one of a handful of people putting their reputation on the line, but that's mostly because that was his schtick in the first place. And even so, his response has been ra…

> Eliezer is one of a handful of people putting their reputation on the line

I have a hard time understanding why anyone takes Yudkowski seriously. What has he done other than found a cult around self-referential ideologies?

By self-referential I mean the ideology's proof rests on its own claims and assertions. Rationalism is rational because it is rational according to its own assertions and methods, not because it has accomplished anything in the real world or been validated in any scientific or empirical-historical way.

Longtermism is particularly inane. It defers everything to a hypothetical ~infinitely large value ~infinitely far in the future, thereby devaluing any real-world pragmatic problems that exist today. War? Climate change? Inequality? Refugee crises? None of that's important compared to the creation of trillions of hypothetical future minds in a hypothetical future utopia whose likelihood we can hypothetically maximize with NaN probability.

You can see how absurd this is by applying it recursively. Let's say we have colonized the galaxy and there are in fact trillions of superintelligent minds living in a pain-free immortal near-utopia. Can we worry about mere proximate problems now? No, of course not. There are countless trillions of galaxies waiting to be colonized! Value is always deferred to a future beyond any living person's time horizon.

The end result of this line of reasoning is the same as medieval religious scholasticism that deferred all questions of human well being to the next world.

I just brought this up to provide one example of the inane nonsense this cult churns out. But what do I know. I obviously have a lower IQ than these people.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#155

Post by ke88y » Mon, Jul 03, 2023, 6:31 PM UTC

arisAlexis wrote: post #145

ke88y wrote: post #134

Earlier quoted context omitted.

> We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021... we found email addresses in papers published at those conferences, in other public data, and in records from our previous survey and Zhang et al 2022. We received 738 responses, some partial, for a 17% response rate. Anyone who has reviewed for, or attended, or just read the author lists of ICML/NeurIPS papers is…

Loling something that serious with several top scientists as you said warning is at least very idiotic regardless where the truth lies. Let's start from that.

I'm laughing at the study methodology, not the underlying topic.

Please don't call me idiotic.

I'd wager I have spent at least 10,000 hours more than you working on (real) safety analyses for (real) AI systems, so refraining from insinuating I'm dismissive would also be nice. But refraining from "idiotic" seems like a minimum baseline.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#156

Post by ALittleLight » Mon, Jul 03, 2023, 6:32 PM UTC

ke88y wrote: post #134

zone411 wrote: post #102

Earlier quoted context omitted.

That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...

> We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021... we found email addresses in papers published at those conferences, in other public data, and in records from our previous survey and Zhang et al 2022. We received 738 responses, some partial, for a 17% response rate. Anyone who has reviewed for, or attended, or just read the author lists of ICML/NeurIPS papers is…

On the one hand, we have "the researchers and PhD's" that you know. On the other, we have published surveys of academics and an open letter from noted experts in the field. One side seems to have much better quality of evidence.

"All the oil and gas engineers I work with say climate change isn't a thing." Hmm, wow, that's really persuasive!

What would you consider evidence of a significant AI risk? From my point of view (x-risk believer) the arguments in favor of existential risk are obvious and compelling. That many experts in the field agree seems like validation of my personal judgement of the arguments. Surveys of researchers likewise seem to confirm this. What evidence do you think is lacking from this side that would convince you?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#157

Post by phillipcarter » Mon, Jul 03, 2023, 6:33 PM UTC

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

I've come to learn that a lot of the rationalist crowd are really just fanfiction authors. That's fine - people should be able to do that - but I don't like how they're given the limelight on things that they generally have little expertise and hands-on knowledge with. I want to like them and want to have them engaged in discourse, but I find them so insufferable.

Not to mention the sub-crowd of rationalists that is weirdly into eugenics. I wish the rest of the rationalist community would disown them loudly.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#158

Post by jrflowers » Mon, Jul 03, 2023, 6:33 PM UTC

llamaimperative wrote: post #141

jrflowers wrote: post #114

Earlier quoted context omitted.

It’s interesting how confidently and obtusely people will proclaim categorical knowledge of the future. It is a little disconcerting that there is a fight between two somewhat cultish sects when it comes to language models. Both sides call them “artificial intelligence”, one side says they’ll save the world, the other side says they’ll end it. There is very little room to even question “Is this actually AI that we’re…

If or when AI is capable of doing all the great things people proclaim it to be able to do, then it will also be capable of doing immense harm, and we should be putting more work into mitigating that. Like it really is that simple. AI generally, LLMs specifically, and certainly this crop of LLMs in particular might end up being inert pieces of technology. But to the precise extent that they are not inert, they carry…

> The AI skeptics don't need to know which outcome the future holds.

But they need to interpret a benign point about the undisputed fact that an LLM has never convinced anybody to fly a plane into a building as some sort of dangerous ignorance of risk that needs correcting.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#159

Post by furyofantares » Mon, Jul 03, 2023, 6:35 PM UTC

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

I'm going to paint three huge demographic swaths here, with the caveat that the merits of any given argument, or any individual's track record, should override anything I say here.

I'm only doing this as a reply to a comment that's also talking about trends among three groups of people.

1. The "people in the trenches" are who I'd least trust about an opinion that everything is OK. Too hard to see the forest for the trees, and too much selection bias.

2. People who gained recognition decades ago, but who are in their slowing-down years as the world continues to change, are among those who I would least trust about an opinion that things are going too fast. It gets harder to keep up as we get older, and as we gain different priorities in life, and I expect this is true no matter how smart we are.

3. People who have spent decades philosophizing about AI-doom are also among those who I would least trust about an opinion that hasn't hugely deviated and become more nuanced as the world has changed and new data has become available.

I am absolutely interested in opinions from all three groups, but the arguments have to stand on their merits. If they're in one of these groups and express the expected opinion, that's actually a strike AGAINST their authority and means the merits of their argument need to be stronger.

I really, really do want to hear opinions from folks in all these groups, I just want to keep this all in mind. I also want to hear opinions from younger philosophers. Folks who are better in-touch with the current world, and rates of progress, and folks who don't have any reputation to uphold.

Also, anyone changing their mind is a big deal. Hofstadter may have changed his mind in the expected direction, but it's still a signal. I'd like to hear more of his thoughts. It doesn't sound carefully considered in the clip in OP's link unfortunately, but that doesn't mean it isn't, and I'd like to hear it.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#160

Post by ALittleLight » Mon, Jul 03, 2023, 6:35 PM UTC

echelon wrote: post #138

zone411 wrote: post #128

Earlier quoted context omitted.

If you read the survey, you'll find that many concerned researchers don't believe we're at a 90% chance of doom, but e.g. 10%. So, this type of response wouldn't be rational if they're thinking that things will go fine most of the time. If these researchers are thinking logically, they would also realize that this kind of reaction has little chance of success, especially if research continues in places like China. It…

> If you read the survey Not a nice way to engage in debate. I've spent more time listening to and refuting these arguments than most. Including debating Eliezer. > many concerned researchers don't believe we're at a 90% chance of doom, but e.g. 10%. A 10% chance of an asteroid hitting the earth would result in every country in the world diverting all of their budgets into building a means to deflect it. > So, this t…

What if some of the experts said "It's really hard to know, but our best guess is there is a 10% chance of this thing hitting the Earth" and other experts said "I really don't think it'll hit the Earth"? My best guess is that Earth wouldn't do much at all about the risk as that seems to be basically what we are facing with AI x-risk.