Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

361–370 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#361

Post by toth »

VincentEvans wrote: post #351
toth wrote: post #282

Earlier quoted context omitted.

It will be very difficult, but if you think about nuclear weapons it doesn't sound so hopeless. First nuclear weapons were used in 1945, almost 80 years ago. Today, out of ~ 200 countries in the world only 9 have nukes. Of those, only 4 did it after Non-proliferation of Nuclear Weapons Treaty. South Africa had them and gave them up under international pressure. Ukraine and other former soviet republics voluntarily ga…

> Ukraine and other former soviet republics voluntarily gave them up after the breakup of the USSR. And no country will ever make the same mistake again. Because the west promised to protect them from aggression. But when the time came to do just that - everyone involved dusted off their copy of Budapest memorandum and handed it over to the lawyers. Sorry for digression. Carry on.

For sure. If the Russian federation breaks up into a bunch of independent states we will have a huge problem.

It was not very fair from civil war already, with the aborted coup. Could you imagine if the Wagner group became a nuclear power??

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#362

Post by jawns »

I am a distributist, of the variety promulgated by G.K. Chesterton and Hillaire Belloc. Their central tenet is that society works best when the ownership of productive property is widely distributed. And the converse is also true: Society does not work well when the ownership of productive property is concentrated in the hands of a small percentage of people, such as an ultra-wealthy ruling class.

That doesn't imply robinhoodism, aka forced redistribution of wealth, but it does imply that economic policy should recognize and be in furtherance of the ideal of widespread ownership.

Back in Belloc and Chesterton's day, "ownership of productive property" meant physical tools and machinery, like farm equipment and printing presses. But in the 21st century, productive property -- that which generates a profit -- is becoming increasingly digital. The general principle still stands, though.

* If you're interested in learning more about distributism as an alternative to capitalism and socialism, I wrote this kids' guide a few years ago, but it's suitable for anyone who's learning more about it: https://shaungallagher.pressbin.com/blog/distributism-for-ki...

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#363

Post by jacquesm »

api wrote: post #349
jacquesm wrote: post #339

Earlier quoted context omitted.

There is no symmetry between destruction and creation/prevention of destruction, destruction is far easier.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes to achieving their stated goals.

Given a modest budget we all had to come up with a plan to destabilize society, 9/11 style attacks, whilst spectacular eventually don't really do a lot of damage, are costly and failure prone though they can definitely drive policy changes and result in a nation doing a lot of harm to itself it will ultimately survive. But what if your goal wasn't to create some media friendly attack but an actual disaster instead, what would it take.

The stories from that night continue to haunt me today. My own solution led to a lot of people going quiet for a bit and contemplating what they could do to defend against it and they realized there wasn't much that they could do, millions if not tens of millions of people would likely die and the budget was under a few hundred bucks. Knowledge about technology is all it takes to do real damage, that, coupled with a lack of restraint and compassion.

The resources are indeed asymmetrical: you need next to nothing to create mass havoc. Case in point: the Kennedy assassination changed the world and the bullet cost a couple of bucks assuming the shooter already had the rifle, and if they didn't it would increase the cost only a tiny little bit.

And you can do far, far worse than that for an extremely modest budget.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#365

Post by PoignardAzur »

RandomLensman wrote: post #207
concordDance wrote: post #200

Earlier quoted context omitted.

Sure, it's not likely to work, but Yud doesn't know of an idea with a better chance of working. If you agree with him on the likelihood of superintelligence in the next few decades and the difficulty of value alignment, what course of action would you suggest?

I think the core is not to agree with the existential risk scenario in the first place. Once you believe some malevolent god will appear and doom us all, anything can be justified (and people have done so in the past - no malevolent god appeared then, btw.).

> I think the core is not to agree with the existential risk scenario in the first place.

I mean, that's motivated reasoning right there, right?

"Agreeing that existential risk is there would lead to complex intractable problems we don't want to tackle, so let's agree there's no existential risk". This isn't a novel idea, it's why we started climate change mitigations 20 years too late.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#366

Post by upupupandaway »

YetAnotherNick wrote: post #256
jeffparsons wrote: post #229

Earlier quoted context omitted.

Sure, I think that kind of "coordination" is a little more credible than what I thought Eliezer was proposing. Maybe I just didn't read carefully enough. One big difference here, however, is that the barrier to entry is lower. You can start doing meaningful AI R&D on existing hardware with _one_ clever person. The same was never true for nuclear weapons, so the pool of even potential players was pretty shallow for a…

> You can start doing meaningful AI R&D on existing hardware with _one_ clever person. With the current hardware, yes. But then again countries could highly restrict all decently powerful hardware and backdoor them, and you need permission from government to run matrix multiplication above few teraflops. Trying to mess with backdoor is a war crime. A weak form of hardware restriction even happens now that Nvidia coul…

And yet: https://www.scmp.com/news/china/science/article/3239998/chin...

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#367

Post by TeMPOraL »

RandomLensman wrote: post #140
vidarh wrote: post #136

Earlier quoted context omitted.

> With threat of military intervention. We've seen how hard it is to do that when the fear is nuclear proliferation. Now consider how hard it is to do that when the research can be done on any sufficiently large set of computational devices, and doesn't even need to be geographically concentrated, or even in your own country. If I was a country wanting to continue AI research under threat of military intervention, I'…

At least nuclear weapons pose an actual existential risk as opposed to AI - and even then we don't just go to war.

Nukes don't have a mind of their own. They're operated by people, who fortunately turned out sane enough that they can successfully threaten each other into a stable state (MAD doctrine). Still, adding more groups to the mix increases risk, which is why non-proliferation treaties are a thing, and are taken seriously.

Powerful enough AI creates whole new classes of risks, but it also magnifies all the current ones. E.g. nuclear weapons become more of an existential risk once AI is in the picture, as it could intentionally or accidentally provoke or trick us into using them.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#368

Post by TeMPOraL »

RandomLensman wrote: post #134
TeMPOraL wrote: post #121

Earlier quoted context omitted.

With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing…

No-one wants a nuclear war over imaginary risks, so not happening.

We better find a way to make it happen, because right now, climate change x-risk mitigation is also not happening, for the same reason.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#369

Post by NoGravitas »

lampington wrote: post #319
zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

> If they are Am I the only one slightly disturbed by the use of "they" in that sentence? I know that the HN commentariat is broad-based, but I didn't realise we already had non-human members ;-)

HN wIlegh tlhInganpu', Human.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#370

Post by PoignardAzur »

persnickety wrote: post #149

> Every new technology is developed and deployed the same way: > You make a prototype, try it at a small scale, make limited deployment, fix the problems, make it safer, and then deploy it more widely. This makes an assumption: that the problematic technology is an intentional development rather than an emergent feature of an intentionally developed technology. We already have a name for such emergent features: "bugs…

> > You make a prototype, try it at a small scale, make limited deployment, fix the problems, make it safer, and then deploy it more widely.

That argument in particular seems like wishful thinking. What in the last 10 years of our tech industry makes him think "move slowly and deploy carefully" is going to be the strategy they adopt now?

And that's without considering the arguments in "We have no moat and neither does OpenAI". No matter how careful the main stakeholders are, once the technology exists, open-source versions will be deployed everywhere soon after with virtually no concern for safety.

Post reply on HN