Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

421–430 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#421

Post by itsoktocry »

zimpenfish wrote: post #260
concordDance wrote: post #199

Earlier quoted context omitted.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

>If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI.

If we go extinct in the next 100 years, it's not going to be from climate change. How would that even work?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#422

Post by toth »

mannykannot wrote: post #379
toth wrote: post #337

Earlier quoted context omitted.

You are right, but "human civilization will be extinct" is an awkward phrase. Human extinction via climate change is off the table. The end of human civilization? Depends on what one means by it. If it means we'll go back to being hunter gatherers it's ridiculous. Some civilizational wide regression is not impossible, but even the direst IUPAC scenario projects only a one digit drop in global GDP (of course, that's t…

> If it means we'll go back to being hunter gatherers it's ridiculous. Indeed, but even going back to the 19th century would have dire consequences, given our current global dependence on the industrial fixing of nitrogen. If civilization were to collapse (and I don't think it will, other than from a near-extinction event), I doubt it would be like going back to any earlier time.

Oh, for sure. I am just saying it's not an extinction risk, not that it is not a huge and pressing problem.

I would still bet against going back to 19th century. Worst case scenario for me is hundreds of millions dying and a big regression in global standards of living, but keeping our level of civilization. Which is horrible, it would be the worst thing that ever happened.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#423

Post by itsoktocry »

jefftk wrote: post #325
zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

>how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

There are thousands of people around the globe working in labs every day that can do this.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#424

Post by jacquesm »

tppiotrowski wrote: post #420
jacquesm wrote: post #363

Earlier quoted context omitted.

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

The people who can do doomsday level harm need to complete a level of education that makes them smart enough and gives them time to consider the implications of their actions. They have a certain level of maturity before they can strike and this maturity makes them understand the gravity of their actions. Also, by this point they are probably set financially due to their education and not upset with the world (possib…

And there are the 'griefers', the people who seem to enjoy watching other people suffer. Unfortunately there are enough of these and they're somewhat organized and in touch with each other.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#425

Post by lettergram »

zimpenfish wrote: post #260
concordDance wrote: post #199

Earlier quoted context omitted.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

zero chance climate change (1-2 degree increase) would end humanity. 5-10 probably wouldn’t end humanity lol

Nuclear war would probably kill 80-90% but even then wouldn’t kill humanity. Projections I’ve seen are only like 40-50% of the countries hit.

AI is scary if they tie it into everything and people slowly stop being capable. Then something happens, and we can’t boil water.

Beyond that scenario I don’t see the risk this century.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#426

Post by toth »

wcoenen wrote: post #352
toth wrote: post #276

Earlier quoted context omitted.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

Climate change and nuclear war are not orthogonal risks. Climate change leads to conflict. For example, the Syria drought of 2006-2010. More climate change leads to larger conflicts, and large conflicts can lead to nuclear exchanges. Think about what happens if India and Pakistan (both nuclear powers) get into a major conflict over water again.

I don't disagree with that, but then the risk is nuclear war. It could be triggered by global warming conflicts or by something else.

Still, like I said somewhere else, even all out nuclear war is unlikely to lead to human extinction. It could push us back to the neolithic in some scenarios, but even then there is some disagreement about what would happen.

Of course, even in the most optimistic scenario it would be really bad and we should do everything we can to avoid it - that goes without saying.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#427

Post by RandomLensman »

ryanklee wrote: post #415
RandomLensman wrote: post #394

Earlier quoted context omitted.

There isn't even agreement on the existence of free will. For non-trivial decisions, let's say some judge's deliberation, to claim it is fully explainable is a stretch.

I didn't say it was fully explainable. And you don't need to settle debates about free will to offer valid causal explanations for human behaviors. Are you saying we don't know at all why anyone does anything they ever do? That every action is totally unpredictable and after the fact that there is no way for us to offer more or less plausible explanations?

We are getting off track in our discussion. For non-trivial decisions we know we can have ex-post justifications from humans but at the same time we know that those aren't necessarily complete or even true at times: A credit officer not liking someone will have a justification for refusal that does not include that and we might never know - from the outside - the true explanation and that credit officer might not even know their biases themselves! AI being explainable is just going beyond what we demand of humans (we demand a justification, not a real explanation).

Also, predictability isn't the same as understanding the full decision process.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#428

Post by itsoktocry »

devnonymous wrote: post #313
toth wrote: post #276

Earlier quoted context omitted.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

Climate change has compounding effects (eg: water scarcity, leading to political conflict, leading to displacement, leading to pandemics ...and that's just one vector). Real world is real real messy. COViD should have taught people that. Unfortunately, it didn't.

>Climate change has compounding effects

Why is the default to assume that every change will be negative?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#429

Post by laverya »

itsoktocry wrote: post #423
jefftk wrote: post #325

Earlier quoted context omitted.

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

> how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

And how many of them are radical religious nutjobs? I think it is plausible that "can spend decades learning a scientific craft" and "wants to kill everyone on the planet" are mutually exclusive personality traits.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#430

Post by TeMPOraL »

vidarh wrote: post #404
TeMPOraL wrote: post #355

Earlier quoted context omitted.

Yes, it's as hard as you describe. But it seems that there are no easier options available - so this is the least hard thing we can do to mitigate the AI x-risk right now.

How do you intend to prevent people from carrying out general purpose computation? Because that is what you'd need to do. You'd need to prevent the availability of any device where users can either directly run software that has not been reviewed, or that can be cheaply enough stripped of CPU's or GPU's that are capable of running un-reviewed software. That review would need to include reviewing all software for "com…

> How do you intend to prevent people from carrying out general purpose computation?

Ask MAFIAA, and Intel, and AMD, and Google, and other major tech companies, and don't forget the banks too. We are already well on our way to that future. Remember Cory Doctorow's "War on general-purpose computation"? It's here, it's happening, and we're losing it.

Therefore, out of many possible objections, this one I wouldn't put much stock in - the governments and markets are already aligned in trying to make this reality happen. Regardless of anything AI-related, generally-available general-purpose computing is on its way out.

Post reply on HN