Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

321–330 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#321

Post by TeMPOraL »

rolisz wrote: post #285
TeMPOraL wrote: post #121

Earlier quoted context omitted.

With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing…

So you're proposing replacing a hypothetical risk (Geoffrey Hinton puts the probability at 1%) with guaranteed pain and destruction, which might escalate to nuclear war? Thank you, but no thank you.

No, I'm proposing replacing a high risk scenario (I disagree with Hinton here) with a low risk scenario, which has low chance to escalate to any serious fighting.

It seems people have missed what the proposed alternative is, so let me spell it out: it's about getting governments to SIGN AN INTERNATIONAL TREATY. It is NOT about having US (or anyone else) policing the rest of the world.

It's not fundamentally more dangerous than the few existing treaties of similar kind, such as those about nuclear non-proliferation. And obviously, all nuclear powers need to be on-board with it, because being implicitly backed by nukes is the only way any agreement can truly stick on the international level.

This level of coordination may seem near-impossible to achieve now, but then it's much more possible than surviving an accidental creation of an AGI.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#322

Post by strken »

logicchains wrote: post #150
atleastoptimal wrote: post #129

Earlier quoted context omitted.

Intelligence needed to be existentially threatening to all human life <<<<<<< intelligence necessary to survey the totality of atoms and particles in the universe. The former is still very possible.

LessWrongers seem to fear a superintelligence that was 95%+ effective at persuading/manipulating humans to achieve whatever its ends were. But this would require the AI being able to predict the future to a degree that wouldn't be remotely possible given the computational power available to it, due to exponentially increasing difficulty.

Why would you need to predict the future to be convincing? Charismatic people aren't acting like oracles, they're just...being charming. They frame actions in ways that make them look good, do little favours for you, stroke your ego, and otherwise have a good grasp of human social dynamics.

Sure, there's an aspect of understanding the other person, but chaos theory doesn't stop politicians from obtaining and keeping power.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#323

Post by api »

Yann is the Phil Zimmerman of our era. He is a hero.

Phil is the author of PGP, the first somewhat usable (by mortals) asymmetric encryption tool. He stood up against governments who wanted to lock down and limit encryption in the 90s, and they of course deployed tons of scare tactics to try to do so.

Yann is fighting that battle today. The Llama models are basically PGP. How he got Meta to pay for them is a story I want to hear. Maybe they just had a ton of GPUs deployed for the metaverse project that were not in use.

If/when I ever finish my current endeavor I’d like to go back to working on AI, which I did way back in college in the oughts. Because of Yann I might be allowed to even if I am not rich or didn’t go to a top ten school.

… because that’s what regulating AI will mean. It will mean it’s only for the right kind of people.

Yann is standing up to both companies intent on regulatory capture and a cult (“rationalism” in very necessary quotes) that nobody would care about had it not been funded by Peter Thiel.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#324

Post by lewhoo »

RandomLensman wrote: post #279
lewhoo wrote: post #273

Earlier quoted context omitted.

I don't know about that. Cloning/modifying humans has potentially so much benefit and yet we seem to stack hypotheses on top of it and conclude it's not worth it. There is also a lot to be said about all the hypotheses of killing all the Hitlers preemptively. I think that whatever we do, we usually arrive at a bit modified trolley problem and try to go from there. We really should never expect that the one particular…

My point was more that it really isn't stretching anything to say: if something bad gets into human reproduction that has unfortunate effects. We can observe that already. So at least the risk case doesn't rely on a lot of hypotheses that might turn out wrong. Whether or not all regulations there are good or optimal is then a separate point again.

I agree but this is too general. If something bad gets into AI (like very biased or false training data) that has unfortunate effects also stands.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#325

Post by jefftk »

zimpenfish wrote: post #260
concordDance wrote: post #199

Earlier quoted context omitted.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#326

Post by api »

JoshTriplett wrote: post #52
lazzlazzlazz wrote: post #35

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

Yudkowski is a crackpot. Nobody would take him seriously if he wasn’t one of Peter Thiel’s pet crackpots.

I should have done a ton more intellectual edgelording in the oughts and grabbed some Thielbucks. I could have recanted it all later.

Of course I probably would have had to at least flirt with eugenics. That seems to be the secret sauce.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#327

Post by TeMPOraL »

concordDance wrote: post #308
shafyy wrote: post #297

Earlier quoted context omitted.

What is ASI and how could it make humans extinct?

Artifical Super Intelligence. Could make humans extinct just like humans could make chimpanzees extinct. Using superior intellect to better leverage available resources than the things it/they are competing with until it out expands/outcompetes them.

Note that ways ASI can make us extinct includes all the ways we could do it to ourselves.

One of the possible scenario is tricking humans into starting WWIII, perhaps entirely by accident. Another is that even being benignly applied to optimize economic activity in broad terms might have the AI strengthen the very failure modes that accelerate climate change.

Point being, all the current global risks won't disappear overnight with the rise of a powerful AI. The AI might affect them for the better, or for worse, but the definition of ASI/AGI is that whichever way it will affect them (or anything in general), we won't have the means to stop it.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#328

Post by varjag »

ben_w wrote: post #246
visarga wrote: post #214

Earlier quoted context omitted.

Yet China and Russia are still getting their chips, by smuggling if need be. So much for the power of blocking access.

China sure, to a degree (though they also make a lot of the phones and therefore just get shipped tons of chips), but wasn't Russia so short they stole washing machines for the chips? Or was that just war propaganda?

It was a meme. You can buy the kind of chips that go into washing machines completely legally and unrestricted from multiple vendors anywhere in the world, and transporting them into russia isn't an issue.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#329

Post by RandomLensman »

lewhoo wrote: post #324
RandomLensman wrote: post #279

Earlier quoted context omitted.

My point was more that it really isn't stretching anything to say: if something bad gets into human reproduction that has unfortunate effects. We can observe that already. So at least the risk case doesn't rely on a lot of hypotheses that might turn out wrong. Whether or not all regulations there are good or optimal is then a separate point again.

I agree but this is too general. If something bad gets into AI (like very biased or false training data) that has unfortunate effects also stands.

But we need to stack a bunch of hypotheses on top of each other to have dangerous AI and those might not be true - that is the difference.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#330

Post by api »

jefftk wrote: post #325
zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

So ask it how to stop someone from doing this. Ask it how to augment the human immune system. Train AIs on the structures of antibiotics and antiviral drugs and have them predict novel ones. Create an AI that can design vaccines against novel agents.

Those things could stop our terrorist and could also stop the next breakout zoonotic virus or “oops!” in a lab somewhere.

Intelligence amplification works for everyone. The assumption you’re making is that it only works for people with ill intent.

The assumption behind all this is that intelligence is on balance malevolent.

This is a result of humans and their negativity bias. We study history and the negative all jumps out at us. The positive is ignored. We remember the holocaust but forget the green revolution, which saved many more people than the holocaust killed. We see Ukraine and Gaza and forget that fewer people per capita are dying in wars today by far than a century ago.

We should be thinking about destructive possibilities of course. Then we should use these intelligence amplifiers to help us prevent them.

Post reply on HN