Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

751–760 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#751

Post by RandomLensman »

pixl97 wrote: post #735
RandomLensman wrote: post #680

Earlier quoted context omitted.

It means intelligence can be killed (and the Khmer were brought down externally). We might not even need to contain, all hypothetical.

We don't have an answer yet to what the form factor of AGI/ASI will be, but if it's anything like current trends the idea of 'killed' is laughable. You can be killed because your storage medium and execution medium are inseparable. Destroy the brain and you're gone, and you don't even get to copy yourself. With AGI/ASI if we can boot it up from any copy on a disk if we have the right hardware then at the end of the d…

No power and it its already dead. Destroy copies, dead. Really not complicated at all.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#752

Post by RandomLensman »

ryanklee wrote: post #729
RandomLensman wrote: post #591

Earlier quoted context omitted.

Because (1) it holds AI to higher standard than humans, (2) it means that even if AI makes better decisions (name your metric), we would deny the use if those decisions if we could not sufficiently explain them. I note, that with certain things we do not do that. For example, with certain pharmaceuticals we were and are quite happy to take the effect even if we do not fully understand how it works.

I'm super confused at how we are disagreeing. Because I agree with all of that.

Nice. Maybe we just talked past each other.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#753

Post by pclmulqdq »

0x00_NULL wrote: post #682
dragonwriter wrote: post #588

Earlier quoted context omitted.

> Every time I see Yudkowsky’s writing I wonder why people take him seriously. Its a perfect thing for rich, extremely privileged people to grab onto as a cause to present thenselves as (and maybe even feel like they are, depending on their capacity for self-delusion) “doing something for humanity” while continuing to ignore the substantive material condition of, as a first order approximation, everyone in the world,…

I lost total interest in every “Stop” argument when Elon Musk started a company to develop AGI during the pause that he signed on to and promoted. Yann LeCunn said that this was just an attempt to seize market share among the big companies and he couldn’t have been more right.

That is transparently what this letter is for most of its signatories. They lost the race (in one way or another) and they want time to catch back up. This is as true for the academics who made big contributions in the past but missed out on the latest stuff as it is for Elon Musk.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#754

Post by pixl97 »

Dig1t wrote: post #131
TeMPOraL wrote: post #121

Earlier quoted context omitted.

With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing…

Are you really suggesting drone striking data centers? What about running AI models on personal hardware? Are we going to require that everyone must have a chip installed in their computer that will report to the government illegal AI software running on the machine? You are proposing an authoritarian nightmare.

Hence the entire point of this discussion... can we avoid the future nightmares that are coming in one way or another? Make AI = nightmare, stop people from making AI = nightmare. The option that seems to be off the table is people going, hell making AI is a nightmare so I won't voluntarily without the need for a police state.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#755

Post by arbitrary_name »

logicchains wrote: post #402
concordDance wrote: post #221

Earlier quoted context omitted.

I'm confused. We know persuasive humans exist, we employ them as politicians or sales people. There are also many unpersuasive humans. Given we seem to have a decent range of persuasiveness even amongst these very very similar minds, why do you think the upper limit for persuasiveness is a charismatic human? Though even if that WAS the limit I'd still be somewhat worried due to the possibility of that persuasiveness…

>Given we seem to have a decent range of persuasiveness even amongst these very very similar minds, why do you think the upper limit for persuasiveness is a charismatic human Because there's a hard limit on how much people can be made to act against their own self interest.

Ever heard of the Jonestown Massacre? Suicide seems to be a fairly consequential upper limit.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#756

Post by RandomLensman »

JoshTriplett wrote: post #645
RandomLensman wrote: post #573

Earlier quoted context omitted.

Even for your last example, two hypotheses need to be true: (1) such information exists, and (2) the AI has access to such information/can generate it. EDIT: actually at least three: (3) the human and/or the AI can apply that information. It also unclear to what extent thinking alone can solve a lot of problems. Similar, it is unclear if humans could not contain superhuman intelligence. Pretty unintelligent humans ca…

> Even for your last example, two hypotheses need to be true: (1) such information exists, and (2) the AI has access to such information/can generate it. EDIT: actually at least three: (3) the human and/or the AI can apply that information. Those trade off against each other and don't all have to be as easy as possible. Information sufficiently dangerous to destroy the world certainly exists, the question is how clos…

We do pretty much nothing to mitigate other, actual extinction level risks - why should AI be special given that its risk has an unknown probability and it could even be zero.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#757

Post by pixl97 »

m_a_g wrote: post #475
hollerith wrote: post #471

Earlier quoted context omitted.

The AGI does not need to control the Pentagon. It might for example invent novel technologies that you and I cannot even imagine because we are much less capable at inventing new technologies than the AGI will be. If members of an alien civilization much more advanced than our own suddenly appeared in orbit around Earth, would that not be dangerous?

That I understand, but my thinking automatically goes to control and restrictions. A superintelligent AGI that's as restricted as chatgpt in terms of what it can actually control seems... not so existentially dangerous? In that scenario, that actual problem would rise by not-so-smart humans listening to that AGI blindly, no?

ChatGPT isn't an ASI.

This said, ChatGPT with a plugin is potentially dangerous. We are lucky enough that it gets caught in loops pretty easily and stops when using tools.

But lets imagine a super intelligence that can only communicate over the internet, what can it accomplish? Are there places you can sign up for banking accounts without a physical presence? If so, then it could open accounts and move money. Money is the ultimate form of instrumental convergence

https://www.youtube.com/watch?v=ZeecOKBus3Q (Robert Miles)

Once you have money you can buy things, like more time on AWS. Or machine parts from some factory in China, or assembly in Mexico. None of these require an in person presence and yet real items exist because of the digital actions. At the end of the day money is all you need, and being super intelligent seems like a good path on figuring out ways to get it.

Oh, lets go even deeper in to things that aren't real and control. Are religions real? If a so called super intelligence can make up its own religion and get believers it becomes the 'follow AGI blindly' scenario you give. It gives me no more comfort that someone could wrap themselves in a bomb vest for a digital voice than it does when they do so for a human one.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#758

Post by RandomLensman »

pixl97 wrote: post #749
RandomLensman wrote: post #397

Earlier quoted context omitted.

I does not, because that kind of AI doesn't exist at the moment and the malevolence etc. are bunch of hypotheses about a hypothetical AI, not facts.

You keep repeating this tired argument in this thread, so just subtract the artificial element from it. Instead imagine a non-human intelligence. Maybe its alien carbon organic. Maybe it's silicon based life. Maybe it's based on electrons and circuits. In this situation, what are the rules of intelligence outside of the container it executes in? Also, every military in the world wargames on hypotheticals because maki…

How would you feel if militaries planned for fighting Egyptian gods? Just because I can imagine something doesn't mean it is real and that it needs planning for. Using effort on imaginary risks isn't free.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#759

Post by RandomLensman »

ryanklee wrote: post #730
RandomLensman wrote: post #581

Earlier quoted context omitted.

I made no such claim. My claim is that it might not be useful to hold AI to a higher standard as humans. With humans we accept certain "whims" in decisions, which is the same as having some unexplainable bit in an AI decision. EDIT: it might not even be useful to insist on explainability if the results are better. We did not and do not do that in other areas.

I noted this elsewhere, but I'll reiterate. I'm confused as to where our disagreement is, because all of that, I agree with. Did I misread your original claims? If so, my apologies for dragging us through the mud.

All good, I think we managed to talk past each other somehow.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#760

Post by mrguyorama »

JoshTriplett wrote: post #497
itsoktocry wrote: post #423

Earlier quoted context omitted.

> how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

There is a massive difference between "there are a few thousand people in the world who might have the specialized knowledge to kill billions" and "anyone with a handful of GPUs or access to a handful of cloud systems could have a conversation with LatestAIUnlimited-900T and make a few specialized mail orders". The Venn diagram of "people who could arrange to kill most of humanity without being stopped" and "people w…

Generating bioweapons is not a trivial task even if you know exactly "how to do it". Ask any biologist how easy it is to grow X your very first attempt.

Then remember buying starter cultures for most bioweapons isn't exactly easy and something anyone is allowed to do.

Even an omniscient AI cannot overcome a skill issue

Post reply on HN