Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

221–230 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#221

Post by concordDance »

logicchains wrote: post #150
atleastoptimal wrote: post #129

Earlier quoted context omitted.

Intelligence needed to be existentially threatening to all human life <<<<<<< intelligence necessary to survey the totality of atoms and particles in the universe. The former is still very possible.

LessWrongers seem to fear a superintelligence that was 95%+ effective at persuading/manipulating humans to achieve whatever its ends were. But this would require the AI being able to predict the future to a degree that wouldn't be remotely possible given the computational power available to it, due to exponentially increasing difficulty.

I'm confused. We know persuasive humans exist, we employ them as politicians or sales people. There are also many unpersuasive humans.

Given we seem to have a decent range of persuasiveness even amongst these very very similar minds, why do you think the upper limit for persuasiveness is a charismatic human?

Though even if that WAS the limit I'd still be somewhat worried due to the possibility of that persuasiveness being used at far greater scale than a human could do...

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#222

Post by YetAnotherNick »

jeffparsons wrote: post #194
upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

Global coordination using coercion has worked for nukes. It's very hard but not impossible to ban NVidia or any competitor and create spy network in all large companies and if any engineer is found to create AI, mark them as terrorist and give them POW treatment. And monitor chip fab like nuclear enrichment facility and launch war with country found to create chip fab, which I belive is hard to do it in complete secrecy.

I believe if top 5 nation agrees for this, it could be acheived. Even with US and China enforcing this, it could likely be acheived.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#223

Post by BlueTemplar »

FeepingCreature wrote: post #206
upupupandaway wrote: post #185

Earlier quoted context omitted.

If good cocaine was only produced in one country with extremely costly machines, a good-enough alternative would appear almost immediately. This has happened millions of times in the history of humanity, I refuse to believe that people haven't learned this lesson.

And yet, it doesn't seem to be that easy with chips.

More because with chips it's still an exponential race. Others will be able to catch up once the race slows down enough.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#224

Post by concordDance »

logicchains wrote: post #107
homefree wrote: post #99

Earlier quoted context omitted.

This is the type of weak dismissal I’m talking about.

A weak dismissal for a weak argument. The strong dismissal is that many processes, even relatively simple ones, are formally chaotic in the https://en.wikipedia.org/wiki/Chaos_theory sense, meaning that predicting how they evolve over time becomes exponentially more difficult the further ahead in time we try to predict. Meaning we quickly get to the point where all the energy in the known universe wouldn't be suffici…

We genuinely don't know what degree of predictive power would be required to wipe out humans so I'm surprised you're willing to make the bold claim that thisnis impossible.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#225

Post by ben_w »

scarmig wrote: post #197
timmytokyo wrote: post #187

Earlier quoted context omitted.

His solution -- assuming it's the same as the one proposed by Yudkowsky, who he quotes -- is launching air strikes against data centers hosting AI computation. So yeah, imaginary.

Ironically, it seems the only way to stop someone from building an omnipotent AI that could destroy the world is to build an omnipotent AI under your control faster than any of the people you need to destroy. If you believe in that sort of thing.

Or possibly just not allow anyone to accumulate enough compute power, given that even the largest model so far still isn't that.

Hence "if you're not willing to go as far as air strikes, you were never serious" or however he phrased it.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#226

Post by BlueTemplar »

esafak wrote: post #78
Aithr wrote: post #74

Earlier quoted context omitted.

"Stop new development." is regulatory capture.

Stop existing development too. Carve an exception for academic research.

«Exception for academic research» seems to be how we got Covid, so...

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#227

Post by atleastoptimal »

logicchains wrote: post #150
atleastoptimal wrote: post #129

Earlier quoted context omitted.

Intelligence needed to be existentially threatening to all human life <<<<<<< intelligence necessary to survey the totality of atoms and particles in the universe. The former is still very possible.

LessWrongers seem to fear a superintelligence that was 95%+ effective at persuading/manipulating humans to achieve whatever its ends were. But this would require the AI being able to predict the future to a degree that wouldn't be remotely possible given the computational power available to it, due to exponentially increasing difficulty.

That's like implying it's impossible for computers to be superhuman at chess given the enormous exponential scales of the number of possible positions. Existing highly persuasive humans can create cults of thousands of people willing to follow their every whim. Why is it so hard to perceive of an agent that tirelessly searches over every human bias and tactic to hone a superhuman level of manipulative skill? You don't need to perfectly predict the future to persuade people of things.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#228

Post by olalonde »

rolisz wrote: post #77
JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

And... How do you do that? Seriously, people have been talking about stoping CO2 emissions for what, 50 years? We've barely made progress there.

This underscores the crux of the issue. Our collective response to curbing emissions has been lackluster, largely due to the fact that the argument for it being beneficial hasn't sufficiently swayed a significant number of people. The case asserting that AI R&D poses an existential risk will be even harder to make. It's hard to enforce a ban when a large part of the population, and perhaps entire countries, do not believe the ban is justified. This contrasts with the near-universal agreement surrounding the prohibition of chemical and biological weapons, or the prevention of nuclear weapons proliferation, where the consensus is considerably stronger.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#229

Post by jeffparsons »

YetAnotherNick wrote: post #222
jeffparsons wrote: post #194

Earlier quoted context omitted.

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

Global coordination using coercion has worked for nukes. It's very hard but not impossible to ban NVidia or any competitor and create spy network in all large companies and if any engineer is found to create AI, mark them as terrorist and give them POW treatment. And monitor chip fab like nuclear enrichment facility and launch war with country found to create chip fab, which I belive is hard to do it in complete secr…

Sure, I think that kind of "coordination" is a little more credible than what I thought Eliezer was proposing. Maybe I just didn't read carefully enough.

One big difference here, however, is that the barrier to entry is lower. You can start doing meaningful AI R&D on existing hardware with _one_ clever person. The same was never true for nuclear weapons, so the pool of even potential players was pretty shallow for a long time, and therefore easier to control.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#230

Post by ben_w »

duckmysick wrote: post #201
timmytokyo wrote: post #187

Earlier quoted context omitted.

His solution -- assuming it's the same as the one proposed by Yudkowsky, who he quotes -- is launching air strikes against data centers hosting AI computation. So yeah, imaginary.

I wonder what would happen if the AI computation is co-hosted in data centers with civilian data.

Then people would have their claims of "bulletproof hosting" and offsite backup policies/business continuity plans tested.
Post reply on HN