Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

201–210 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#201

Post by duckmysick »

timmytokyo wrote: post #187
topspin wrote: post #142

Earlier quoted context omitted.

> "Stop" is not regulatory capture. "Stop" is a fiction that exists exclusively inside your head. Unless your solution anticipates a planet wide GULAG system run by a global government your "stop" is infeasible.

His solution -- assuming it's the same as the one proposed by Yudkowsky, who he quotes -- is launching air strikes against data centers hosting AI computation. So yeah, imaginary.

I wonder what would happen if the AI computation is co-hosted in data centers with civilian data.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#202

Post by jstanley »

scarmig wrote: post #197
timmytokyo wrote: post #187

Earlier quoted context omitted.

His solution -- assuming it's the same as the one proposed by Yudkowsky, who he quotes -- is launching air strikes against data centers hosting AI computation. So yeah, imaginary.

Ironically, it seems the only way to stop someone from building an omnipotent AI that could destroy the world is to build an omnipotent AI under your control faster than any of the people you need to destroy. If you believe in that sort of thing.

The only thing that can stop a bad guy with AI is a good guy with AI.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#203

Post by concordDance »

3rd3 wrote: post #170
TeMPOraL wrote: post #121

Earlier quoted context omitted.

With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing…

Since when are military spooks and political opportunists better at deciding on our technological future than startups and corporations? The degree of global policing and surveillance necessary to fully prevent secret labs from working on AI would be mind-boggling . How would you ensure all government actors are sticking to the same safety standards rather than seizing power by implementing AI hastily? This problem h…

> The degree of global policing and surveillance necessary to fully prevent secret labs from working on AI would be mind-boggling.

It's not that bad given the compute requirements for training even the basic LLMs we have today.

But yes, it's a long shot.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#204

Post by neilkk »

candiodari wrote: post #143
wood_spirit wrote: post #122

Earlier quoted context omitted.

You can’t _stop_ it. You can make it illegal in some countries. But outlawing it just means some obvious candidate countries or private actors develop it first instead.

I think these attempts will simply point out that math is a higher power than law. If something becomes possible, and becomes easy, there is nothing law can do to stop it. Just look at the attempts to stop drugs in North West Europe. Huge sentences (so huge they are in fact never fully carried out), tens of dead police officers on a yearly basis, triple that in civilian victims and drugs are easier to get than ever b…

> tens of dead police officers on a yearly basis

What are you talking about? This does not happen in North West Europe.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#205

Post by concordDance »

vidarh wrote: post #136
TeMPOraL wrote: post #121

Earlier quoted context omitted.

With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing…

> With threat of military intervention. We've seen how hard it is to do that when the fear is nuclear proliferation. Now consider how hard it is to do that when the research can be done on any sufficiently large set of computational devices, and doesn't even need to be geographically concentrated, or even in your own country. If I was a country wanting to continue AI research under threat of military intervention, I'…

The big cloud providers would be obligated to ensure their systems were nor being used for AI training.

Yes, this would be a substantial regulatory burden.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#206

Post by FeepingCreature »

upupupandaway wrote: post #185
FeepingCreature wrote: post #154

Earlier quoted context omitted.

The aim is to stop it from becoming easy. The drug war would look very different if good cocaine was only produced in one country with extremely costly chemistry machines from one company.

If good cocaine was only produced in one country with extremely costly machines, a good-enough alternative would appear almost immediately. This has happened millions of times in the history of humanity, I refuse to believe that people haven't learned this lesson.

And yet, it doesn't seem to be that easy with chips.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#207

Post by RandomLensman »

concordDance wrote: post #200
upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

Sure, it's not likely to work, but Yud doesn't know of an idea with a better chance of working. If you agree with him on the likelihood of superintelligence in the next few decades and the difficulty of value alignment, what course of action would you suggest?

I think the core is not to agree with the existential risk scenario in the first place.

Once you believe some malevolent god will appear and doom us all, anything can be justified (and people have done so in the past - no malevolent god appeared then, btw.).

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#208

Post by homefree »

somenameforme wrote: post #161
homefree wrote: post #70

I think Yann is probably wrong. He refuses to engage earnestly with the “doomer” arguments. The same type of motivated reasoning could also be attributed to himself and Meta’s financial goals - it’s not a persuasive framing. The attempts I’ve seen from him to discuss the issue that aren’t just name calling are things like saying he knows smart people and they aren’t president - or even that his cat is pretty smart an…

The "doomer" arguments can be dismissed relatively easily by one logical consideration. In each country there is one group far more dangerous than any other. This group tends to have 'income' in the billions to hundreds of billions of dollars. And this money is exclusively directed towards finding new ways to kill people, destroy governments, and generally enable one country to forcibly impose their will on others, w…

This is missing the point - that you’re not in control of the superintelligent AGI. The human usage is not the e-risk.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#209

Post by YetAnotherNick »

topspin wrote: post #142
JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

> "Stop" is not regulatory capture. "Stop" is a fiction that exists exclusively inside your head. Unless your solution anticipates a planet wide GULAG system run by a global government your "stop" is infeasible.

"Stop" might be possible if doing AI is expensive enough like say biotech. Currently it is in the limit of being expensive(costing few hundred thousands to few million for state of the art model), but it is very likely to change in near future that will change as even currently most of the cost comes not from energy of computation, but from expensive NVIDIA hardware.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#210

Post by jeffparsons »

concordDance wrote: post #199
jeffparsons wrote: post #194

Earlier quoted context omitted.

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

It can't be the only card left to play. There are so many others. They may be weak, but they still have to be better than one that is guaranteed to be absolutely useless.

Example of a weak, but at least _better_ option: convince _one_ government (e.g. USA) that it is in their interest to massively fund an effort to (1) develop an AI that is compliant to our wishes and can dominate all other AIs, and (2) actively sabotage competing efforts.

Governments have done much the same in the past with, e.g. conventional weapons, nuclear weapons, and cryptography, with varying levels of success.

If we're all dead anyway otherwise, then I don't see how that can possibly be a worse card.

Edit: or even try to convince _all_ governments to do this so that they come out on top. At least then there's a greater chance that the bleeding edge of this tech will be under the stewardship of a deliberate attempt for a country to dominate the world, rather than some bored kid who happens to stumble upon the recipe for global paperclips.

Post reply on HN