Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

351–360 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#351

Post by VincentEvans »

toth wrote: post #282
upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

It will be very difficult, but if you think about nuclear weapons it doesn't sound so hopeless. First nuclear weapons were used in 1945, almost 80 years ago. Today, out of ~ 200 countries in the world only 9 have nukes. Of those, only 4 did it after Non-proliferation of Nuclear Weapons Treaty. South Africa had them and gave them up under international pressure. Ukraine and other former soviet republics voluntarily ga…

> Ukraine and other former soviet republics voluntarily gave them up after the breakup of the USSR.

And no country will ever make the same mistake again. Because the west promised to protect them from aggression. But when the time came to do just that - everyone involved dusted off their copy of Budapest memorandum and handed it over to the lawyers.

Sorry for digression. Carry on.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#352

Post by wcoenen »

toth wrote: post #276
zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

Climate change and nuclear war are not orthogonal risks.

Climate change leads to conflict. For example, the Syria drought of 2006-2010.

More climate change leads to larger conflicts, and large conflicts can lead to nuclear exchanges. Think about what happens if India and Pakistan (both nuclear powers) get into a major conflict over water again.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#353

Post by TeMPOraL »

sebastianz wrote: post #316
TeMPOraL wrote: post #307

Earlier quoted context omitted.

I'm Polish. My country is the one whose highest distinction on the international level was being marked by the US for preemptive glassing, to make it more difficult for the Soviets to drive their tanks west. Which is completely irrelevant to the topic at hand anyway.

> Which is completely irrelevant to the topic at hand anyway. It's not irrelevant. I was implying that you did not specify who is doing this military intervention that you see as a solution. What are their values, who decides the rules that the rest of the world will have to follow, with what authority, and who (and how) will the policing be done.

> I was implying that you did not specify who is doing this military intervention that you see as a solution. What are their values, who decides the rules that the rest of the world will have to follow, with what authority, and who (and how) will the policing be done.

Like nuclear non-proliferation treaties or various international bans on bioweapons, but more so. The idea is that humanity is racing full steam ahead into an AI x-risk event, so it needs to slow down, and the only way it can slow down is through an international moratorium that's signed by most nations and treated seriously, where "seriously" includes the option of signatories[0] authorizing a military strike against a facility engaged in banned AI R&D work.

In short: think of UN that works.

--

[0] - Via some international council or whatever authority would be formed as part of the treaty.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#354

Post by upupupandaway »

toth wrote: post #282
upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

It will be very difficult, but if you think about nuclear weapons it doesn't sound so hopeless. First nuclear weapons were used in 1945, almost 80 years ago. Today, out of ~ 200 countries in the world only 9 have nukes. Of those, only 4 did it after Non-proliferation of Nuclear Weapons Treaty. South Africa had them and gave them up under international pressure. Ukraine and other former soviet republics voluntarily ga…

So... 44% of countries that have obtained nuclear weapons did so after the Nuclear Weapons Treaty. China alone added 60 warheads in 2022. It doesn't matter that it works 90% of the time - if a single entity is able to break out of the prohibition, it's game over.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#355

Post by TeMPOraL »

vidarh wrote: post #136
TeMPOraL wrote: post #121

Earlier quoted context omitted.

With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing…

> With threat of military intervention. We've seen how hard it is to do that when the fear is nuclear proliferation. Now consider how hard it is to do that when the research can be done on any sufficiently large set of computational devices, and doesn't even need to be geographically concentrated, or even in your own country. If I was a country wanting to continue AI research under threat of military intervention, I'…

Yes, it's as hard as you describe. But it seems that there are no easier options available - so this is the least hard thing we can do to mitigate the AI x-risk right now.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#356

Post by api »

toth wrote: post #276
zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

> There is no scenario where climate change leads to human extinction

https://en.m.wikipedia.org/wiki/Clathrate_gun_hypothesis

May not be possible with modern deposits but I don’t think we are 100% sure of that, and you asked.

We could also probably burn enough carbon to send CO2 levels up where they would cause cognitive impairment to humans if we really didn’t give a damn. That would be upwards of 2000ppm. We probably have enough coal for that if we decide we don’t need Alaska. Of course that would be even more of a species level Darwin Award because we’d see that train coming for a long time.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#357

Post by candiodari »

dnissley wrote: post #299
upupupandaway wrote: post #185

Earlier quoted context omitted.

If good cocaine was only produced in one country with extremely costly machines, a good-enough alternative would appear almost immediately. This has happened millions of times in the history of humanity, I refuse to believe that people haven't learned this lesson.

People seem to have very small imaginations when it comes to thinking about how they might circumvent the restrictions they are proposing. I don't know of it's just wishful thinking, desperation, or what. For example if OpenAI et al were reduced to using consumer gpus to run their training loads it would increase costs, but not by an order of magnitude. It would just be one more hurdle to overcome. And there are stil…

As someone who works in medical research, I'm pretty sure it's possible to create a GM bacterial version of poppy's, just like we've got that for insulin, that would end the drug wars, by enabling hundreds of grams of the stuff to be produced in a single bottle that can be kept virtually anywhere (it has to be kept heated, and shaken from time to time).

Such a bottle's contents can be trivially duplicated and the whole process is essentially the same as the first steps in brewing beer.

Doing it for cocaine is simpler than doing it for insulin, which is in one university a second-year exercise. I don't think it'll be that long before this happens.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#358

Post by upupupandaway »

visarga wrote: post #214
FeepingCreature wrote: post #206

Earlier quoted context omitted.

And yet, it doesn't seem to be that easy with chips.

Yet China and Russia are still getting their chips, by smuggling if need be. So much for the power of blocking access.

That's how North Korea got nuclear weapons. It's fascinating that people don't understand this.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#359

Post by TeMPOraL »

vidarh wrote: post #258
concordDance wrote: post #205

Earlier quoted context omitted.

The big cloud providers would be obligated to ensure their systems were nor being used for AI training. Yes, this would be a substantial regulatory burden.

It'd be more than a substantial regulatory burden, it's impossible. At most they'd be able to drive up the cost by forcing users to obscure it by not doing full sets of calculations at a single provider. Any attempt even remotely close to extreme enough to have any hope of being effective would require such invasive monitoring of computations run that it'd kill their entirely cloud industry. Basically, you'll only su…

Even if it's this costly, it's still possible, and much cheaper than the consequences of an unaligned AGI coming into existence.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#360

Post by alephnan »

rpigab wrote: post #294
RandomLensman wrote: post #148

Earlier quoted context omitted.

If every investment became risk free because of perfect information about the future, it would likely increase the total market value by quite a lot.

Or decrease, because who would go to the casino if everyone knew the outcome? Only the owners and staff. "Useless" companies are part of the total market value. If they never get any funding, that's less value overall. Even if that translates to more value for "useful" companies. Also, if you know with absolute certainty from the beginning that Apple Inc is going to be worth X billion dollars, then you never get to b…

People still buy airline stocks
Post reply on HN