Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

391–400 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#391

Post by arisAlexis »

ke88y wrote: post #167
arisAlexis wrote: post #162

Earlier quoted context omitted.

[flagged]

> I repeat everyone that is loling at x-risk an idiot I mean this is the nicest possible way and hope that you consider it constructive: you're willfully misconstruing what I am saying and then calling me an idiot. This isn't a nice thing to do and shuts down the conversation. If I believed you were operating in good faith, I might take the time to explain why my practical/scientific experience makes me incredulous o…

> you're willfully misconstruing what I am saying and then calling me an idiot

No I am not, you put yourself in the group that makes x-risk sound ridiculous on purpose. If you do that yes. Can you make yourself a third person listening to someone saying: "important people said there is a chance everyone dies but I am laughing at them"? How psychopathic is this?

> If I believed you were operating in good faith

No, you are right. I have a back agenda, it's a dark one. It is about not everyone dying. Call me a backstabbing lying scam.

> why my life experience makes me suspicious of the personalities involved in pushing AI x-risk

Mistake no1: "pushing x-risk" makes no sense. Being afraid that we will die makes a lot of sense, humans are built this way. Some take action some laugh etc. Mistake no2: Hinton, Bengio, Ilya Sutsekever that is the guys that built the technology you spent thousands of hours researching as you said. If you are suspicious of these guys, just get out of this field, it's not for you clearly. That would be insane, a physicist being suspicious of Feynman or Bohr.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#392

Post by [deleted] »

api wrote: post #154
echelon wrote: post #106

Earlier quoted context omitted.

If these people were so concerned, they'd by shouting from the hilltops and throwing their entire life savings into stopping us. They would organize workplace walkouts and strikes. There would be protests and banners. Burning data centers. Eliezer is one of a handful of people putting their reputation on the line, but that's mostly because that was his schtick in the first place. And even so, his response has been ra…

> Eliezer is one of a handful of people putting their reputation on the line I have a hard time understanding why anyone takes Yudkowski seriously. What has he done other than found a cult around self-referential ideologies? By self-referential I mean the ideology's proof rests on its own claims and assertions. Rationalism is rational because it is rational according to its own assertions and methods, not because it…

[deleted]

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#393

Post by bangkoksbest »

ALittleLight wrote: post #358
ke88y wrote: post #355

Earlier quoted context omitted.

> Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't show anything. Of course. I was merely explaining the reason why my response would focus on x-risk rather than better surveys, and why. > but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested…

The reason you aren't persuasive is because you just ignore arguments without responding to them. At least, you've done so here - perhaps you've responded other places with your decade of experience. When presented with surveys and claims of experts, you just dismiss those out of hand "I'm an expert too, therefore ignore other experts." When presented with an argument you just say "That's religious dogma and I reject…

Those first two points are pretty wild assertions that are offered without much compelling evidence.

The final two points are more plausible, although the last one is sort of tautological since any risk is by definition something we aren't certain won't happen. However part of their plausibility as risks is because the first two points are not anywhere near our current state, and therefore it's unclear to us how to clearly evaluate risks that are based on unknown and possibly fictional contexts.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#394

Post by mnky9800n »

ballenf wrote: post #360
TedDoesntTalk wrote: post #350

Earlier quoted context omitted.

Here is ChatGPT 4’s summary of a sequel. I don’t think it’s very good. Maybe another AI would enjoy it though. > Create a summary of a sequel to the movie “War Games” Title: "War Games: Firewall" In "War Games: Firewall", now-adult computer genius David Lightman (Matthew Broderick) has been working as a cyber security consultant for the Pentagon. When an advanced, multi-national cyber-terrorist organization emerges,…

That sounds better than every recent sequel coming out of Hollywood, imho. Would watch.

It's very 2005. It's surprising this movie wasn't made.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#395

Post by ALittleLight »

bangkoksbest wrote: post #393
ALittleLight wrote: post #358

Earlier quoted context omitted.

The reason you aren't persuasive is because you just ignore arguments without responding to them. At least, you've done so here - perhaps you've responded other places with your decade of experience. When presented with surveys and claims of experts, you just dismiss those out of hand "I'm an expert too, therefore ignore other experts." When presented with an argument you just say "That's religious dogma and I reject…

Those first two points are pretty wild assertions that are offered without much compelling evidence. The final two points are more plausible, although the last one is sort of tautological since any risk is by definition something we aren't certain won't happen. However part of their plausibility as risks is because the first two points are not anywhere near our current state, and therefore it's unclear to us how to c…

I think it's surprising that you think the first two points are controversial. They seem quite obvious to me.

First, we know of no special requirement that exists in human brains that provides intelligence that machines lack. In other words, We don't have any reason to expect that machines will be limited by human intelligence or some level below. On the contrary, we have great reasons to expect that machines will easily be able to exceed human intelligence - for example, the speed and reliability of digital computation or the fact that computers can be arbitrary sizes and use amounts of power and dump waste heat that our biological brains couldn't dream of. If you accept that AI is making progress along a spectrum of intelligence, moving closer towards human level intelligence now, then it seems absurd to doubt that it would be possible for AI to surpass human intelligence. Why would that be impossible? It's like claiming we could never build a machine larger than a human or stronger than a human. There's no reason or evidence to support such a claim.

Second is the idea that AI is making progress towards human and superhuman intelligence. I think you should be convinced of this by simply looking at the state of the art 5 years ago versus today. If you put those points on a plane and draw a line between them, where is that line in 5 years or 10?

Today GPT4 can play chess, write poetry, take standardized tests and do pretty well, answer math problems, write code, tell jokes, translate languages, and just generally do all sorts of cognitive tasks. Capabilities like these did not exist five years ago or to the extent they did, they existed only in rudimentary forms compared to what GPT4 is capable of. We can see similar progress in different domains, not just large language models - for example, image generation or recognition.

Progress might not continue then again it might. It might accelerate. As the capabilities of the models increase, they might contribute to accelerating the progress of artificial intelligence.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#396

Post by ALittleLight »

rixed wrote: post #372
ALittleLight wrote: post #358

Earlier quoted context omitted.

The reason you aren't persuasive is because you just ignore arguments without responding to them. At least, you've done so here - perhaps you've responded other places with your decade of experience. When presented with surveys and claims of experts, you just dismiss those out of hand "I'm an expert too, therefore ignore other experts." When presented with an argument you just say "That's religious dogma and I reject…

Could you elaborate on 3? It's easy to envision how AI can be used by $powerful to improve its control of $powerless, even before AGI, but how an AI would grab power from $powerful and then behave even worse is harder to imagine.

Remember that an AI doesn't have to grab power or be sentient or be an agent of any kind. AI could lead to the destruction or domination of humanity, even if AI is simply a passive tool completely obedient to its operators.

Suppose GPT5 or 6 is multimodal and extremely intelligent. Any work that could be done remotely at a computer could be done by the superhuman intelligence of GPT 6. This is everything from creating music and movies, virtual YouTubers and live streamers, to customer support agents, software developers and designers, most legal advice, and many more categories besides. Providing all of these jobs at extremely low cost will make Open AI exceedingly rich - what if they want to get richer? How about robotics? The AI can help them not only design and iterate and improve on robots and robot hardware, but the AI can also operate the robots for them. Now they can not only do the kind of jobs that can be done on the computer. They can do the jobs that require physical manipulation as well and that category of jobs extends to things like security, military, police.

At this point there are at least two possibilities. First is that the AI is effectively controlled by OpenAI in which case the decision makers that open AI would have effective and lasting control over all of humanity. No plucky band of rebels could possibly overthrow the AI powered tyranny - That's the strength of relentless super intelligence. The second possibility is that open AI doesn't have it well controlled - that could mean bugs or unexpected behavior or it could mean the model expressing some kind of agency of its own - think of the things that the Bing AI was saying before Microsoft got it mostly under control. If the AI isn't under control, it may choose to eliminate humanity simply because we might be in its way or we might be an impediment to its plans.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#397

Post by m463 »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

I suspect that nobody accepts risks until they are realized in some way.

Nobody really did anything about air pollution until you could see smog, and then catalytic converters were mandated.

Lots of people complained about privacy, but because for practical purposes nobody can "see" a loss of privacy, very little progress has been made.

Not many self-driving car fatalities, so the loud pundit proclamations have not gotten much traction.

Now AI... will this take the path of privacy, where nobody can see it, so nothing happens?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#398

Post by Borrible »

mnky9800n wrote: post #202
throwway120385 wrote: post #180

Earlier quoted context omitted.

In my opinion the weirdness is that everyone is talking about how AI is so much better than humans and we're all going to get replaced. But almost nobody is talking about how tech companies will use this new technology to abdicate responsibility for the decisions they undertake. If you thought redlining was bad, you should see how mortgage companies treated people when they used an AI to accomplish the same goals. No…

As well as these models will all be used to launder content. In the near future you will likely be able to ask an AI to create the next 15 sequels to the movie war games starring a young matthew broderick. Episode 12 will have a karate kid cross over. But how much of what is there will be an actually new thing? How much will be scraped from my War Games fan fic website that contains scripts of 27 sequels to War Games…

Like the Star Wars franchise?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#399

Post by Borrible »

akomtu wrote: post #357
maizeq wrote: post #220

Earlier quoted context omitted.

I work in a AI research lab for a large tech company and my observations are completely opposed to yours. Almost every researcher I have spoken believes that real risk exists, to some degree or other. Recent surveys of people in industry have largely borne this out - your anecdote sounds more like an anomaly to me.

It doesn't take a PhD to see where it goes. Once good enough AI is built, and we're close to that, it will be replicated by China, Russia and other big dictatorships. What would Russian dictator give for an AI that can lead his army with surgical precision? He would do anything. Other nations will have to follow. There won't be benevolent committees deciding how to use AI for good, it will be about survival. And it's…

>What would Russian dictator give for an AI that can lead his army with surgical precision

Still needs surgical instruments.

But more important, patients that do not defend themselves with claws, teeth and AI.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#400

Post by dlkf »

lispm wrote: post #389
dlkf wrote: post #384

Earlier quoted context omitted.

What did he get right? While Hinton, LeCun, and others were laying the foundations of deep learning in the 80s, Hofstadter was writing expert systems in Lisp.

Just for the record: Hinton was a Lisp user at some point in time. 1984 he got a Symbolics 3600. LeCun co-wrote Lush, a specialized Lisp for numerical computing. https://lush.sourceforge.net

That you have to reach for such obscure examples kind of illustrates my point, no? Geoff Hinton isn’t known for ”using Lisp at some earlier point in time.”

It’s not my goal to denigrate Lisp. I think Lisp is great. I learned programming with Dr Racket. But Hofstadter’s contribution to our understanding of intelligence is somewhere between ”negligible” and ”counterproductive.”

Post reply on HN