Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

411–420 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#411

Post by jacquesm »

Teever wrote: post #407
zarzavat wrote: post #374

Earlier quoted context omitted.

A real world taste of what happens when smart terrorists decide to attack: https://en.m.wikipedia.org/wiki/Tokyo_subway_sarin_attack Chemical weapons are absolutely terrifying , especially the modern ones like VX. In recent years they have mostly been used for targeted assassinations by state actors (Kim Jong Nam, Salisbury Novichok poisonings). If AI makes this stuff easier to carry out then we are completely fucked…

Imagine an attack with mass produced prions that are desseminated in the food chain. By the time people start to realize what's going on it will be too late.

You could just mix it in with animal feed... don't give any militant vegetarians ideas now.

Not that the meat industry would ever score an own goal like that.

https://en.wikipedia.org/wiki/Bovine_spongiform_encephalopat...

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#412

Post by nmca »

lazzlazzlazz wrote: post #35

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

None of the world-leading scientists that co-authored this paper:

https://managing-ai-risks.com/

Work for big labs? How is that you think the proposed conspiracy works exactly? They're paying off Yoshua Bengio?

(disclaimer: I work on safety for a large lab. But none of these proponents do!)

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#413

Post by CuriouslyC »

toth wrote: post #276
zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

There is a scenario. The climate has been buffering global warming, and as its ability to do that fails, the rate of temperature increase accelerates along with many extreme climate events. This rapid acceleration and series of catastrophes causes a combination of mass migrations along with unprecedented crop failures. First world countries are destabilized by sky high food prices and the mass influx of migrants, leading to a big uptick in wars and revolutions. Eventually mass starvation pushes us into wars for food that quickly spiral into mutual annihilation.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#414

Post by PeterStuer »

logicchains wrote: post #114
PeterStuer wrote: post #100

Earlier quoted context omitted.

Would that still not in the above toy example lead to 0$ vs 2.000.000$/(some factor)?

Yes, but it's not going to have any meaningful impact on the overall gdp because it couldn't happen at scale (requires one party to have 1.000.000$ to spend in the first place).

But isn't this exactly what has been happening on a macro scale with the 'service economy'? We drew ever larger segments of people and activities into the world of financial transactions because of the massive amount of rentseeking money to be made in this environment. We doubled the amount of time the average family had to spend in salaried activity, and we passed start twice as now all activities that were previously selfserviced were also 'market' procured as no-one had time/energy left to take care of this after a long workday.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#415

Post by ryanklee »

RandomLensman wrote: post #394
ryanklee wrote: post #344

Earlier quoted context omitted.

That's a bit of an exaggeration. We understand high level motivations as being highly correlated to certain outcomes. Hunger, eat. Horny, sex. Wronged, vengeance. Money, directed. You don't necessarily need to know the underlying processes to make reliable predictions or valid causal explanations about human behavior.

There isn't even agreement on the existence of free will. For non-trivial decisions, let's say some judge's deliberation, to claim it is fully explainable is a stretch.

I didn't say it was fully explainable. And you don't need to settle debates about free will to offer valid causal explanations for human behaviors.

Are you saying we don't know at all why anyone does anything they ever do? That every action is totally unpredictable and after the fact that there is no way for us to offer more or less plausible explanations?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#416

Post by ertgbnm »

pjc50 wrote: post #340
timmytokyo wrote: post #187

Earlier quoted context omitted.

His solution -- assuming it's the same as the one proposed by Yudkowsky, who he quotes -- is launching air strikes against data centers hosting AI computation. So yeah, imaginary.

Ah, so we're going to start the global nuclear exchange with China based on the paranoid fantasies of some guy on Twitter? And this is supposed to be taken seriously?

And world war two was started by some failed artist from Germany. Deliberately reducing things is easy and does not prove anything.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#417

Post by toth »

api wrote: post #356
toth wrote: post #276

Earlier quoted context omitted.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

> There is no scenario where climate change leads to human extinction https://en.m.wikipedia.org/wiki/Clathrate_gun_hypothesis May not be possible with modern deposits but I don’t think we are 100% sure of that, and you asked. We could also probably burn enough carbon to send CO2 levels up where they would cause cognitive impairment to humans if we really didn’t give a damn. That would be upwards of 2000ppm. We proba…

Interesting, I have some vague recollections off hearing about this.

But according to your link, IPCC says it's not a plausible scenario in the medium term. And I'd say that even 8 degrees of warming wouldn't be enough for extinction or end of human civilization. But there it could be double digit percentage of human population dying.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#418

Post by toth »

devnonymous wrote: post #313
toth wrote: post #276

Earlier quoted context omitted.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

Climate change has compounding effects (eg: water scarcity, leading to political conflict, leading to displacement, leading to pandemics ...and that's just one vector). Real world is real real messy. COViD should have taught people that. Unfortunately, it didn't.

I am not saying it's not a huge problem. Hundreds of millions dying would probably make it the worst thing that ever happened.

I am just saying it's not an extinction risk.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#419

Post by [deleted] »

TeMPOraL wrote: post #386
api wrote: post #349

Earlier quoted context omitted.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…

> The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. You're forgetting about governments and militaries of major powers. It's not that they want to burn the world for no reason - but they still end up seeking capability to do doomsday levels of harm, by continuously seeking to have margin…

[deleted]

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#420

Post by tppiotrowski »

jacquesm wrote: post #363
api wrote: post #349

Earlier quoted context omitted.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

The people who can do doomsday level harm need to complete a level of education that makes them smart enough and gives them time to consider the implications of their actions. They have a certain level of maturity before they can strike and this maturity makes them understand the gravity of their actions. Also, by this point they are probably set financially due to their education and not upset with the world (possibly the case for you and friends)

Then there are the script kiddies that find a tool online that someone smarter than them wrote and deploy it to wreak havoc. The script kiddies are the people I worry about. They don't have the maturity of doing the work and the emotional stability of older age and giving them something powerful through AI worries me.

Theorem: by the time someone reaches the intelligence level required to annihilate the world they can comprehend the implications of their actions.

Post reply on HN