Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

251–260 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#251

Post by [deleted] »

TeMPOraL wrote: post #121
rolisz wrote: post #77

Earlier quoted context omitted.

And... How do you do that? Seriously, people have been talking about stoping CO2 emissions for what, 50 years? We've barely made progress there.

With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing…

[deleted]

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#252

Post by reducesuffering »

persnickety wrote: post #149

> Every new technology is developed and deployed the same way: > You make a prototype, try it at a small scale, make limited deployment, fix the problems, make it safer, and then deploy it more widely. This makes an assumption: that the problematic technology is an intentional development rather than an emergent feature of an intentionally developed technology. We already have a name for such emergent features: "bugs…

Yep. And there are already 20 year old viruses out there, autonomously following their programming, causing issues aligned with no human, as the original humans do not profit off it anymore, and unable to “be turned off”…

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#253

Post by brutusborn »

RandomLensman wrote: post #196
brutusborn wrote: post #135

Earlier quoted context omitted.

What makes it unlikely?

Because everyone getting richer from removing uncertainty isn't a risk to start with.

What about a single actor getting rich and leaving everyone else’s certainty unchanged?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#254

Post by richardw »

logicchains wrote: post #150
atleastoptimal wrote: post #129

Earlier quoted context omitted.

Intelligence needed to be existentially threatening to all human life <<<<<<< intelligence necessary to survey the totality of atoms and particles in the universe. The former is still very possible.

LessWrongers seem to fear a superintelligence that was 95%+ effective at persuading/manipulating humans to achieve whatever its ends were. But this would require the AI being able to predict the future to a degree that wouldn't be remotely possible given the computational power available to it, due to exponentially increasing difficulty.

The AI can run an A/B test, no? It doesn't need to predict perfectly, it needs a goal and the ability to test and measure, and adapt as things change.

Generative AI creating text/audio/video with a goal and a testing/feedback loop. I'm not an AI and I think I could make it work given a small amount of resources.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#255

Post by concordDance »

jeffparsons wrote: post #210
concordDance wrote: post #199

Earlier quoted context omitted.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

It can't be the only card left to play. There are so many others. They may be weak, but they still have to be better than one that is guaranteed to be absolutely useless. Example of a weak, but at least _better_ option: convince _one_ government (e.g. USA) that it is in their interest to massively fund an effort to (1) develop an AI that is compliant to our wishes and can dominate all other AIs, and (2) actively sabo…

That depends on making powerful AIs and being sure they are well aligned being as easy as making powerful AIs which look aligned to cursory inspection. The former seems much harder, though I'd say talk to Robert E. Miles for a more detailed look.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#256

Post by YetAnotherNick »

jeffparsons wrote: post #229
YetAnotherNick wrote: post #222

Earlier quoted context omitted.

Global coordination using coercion has worked for nukes. It's very hard but not impossible to ban NVidia or any competitor and create spy network in all large companies and if any engineer is found to create AI, mark them as terrorist and give them POW treatment. And monitor chip fab like nuclear enrichment facility and launch war with country found to create chip fab, which I belive is hard to do it in complete secr…

Sure, I think that kind of "coordination" is a little more credible than what I thought Eliezer was proposing. Maybe I just didn't read carefully enough. One big difference here, however, is that the barrier to entry is lower. You can start doing meaningful AI R&D on existing hardware with _one_ clever person. The same was never true for nuclear weapons, so the pool of even potential players was pretty shallow for a…

> You can start doing meaningful AI R&D on existing hardware with _one_ clever person.

With the current hardware, yes. But then again countries could highly restrict all decently powerful hardware and backdoor them, and you need permission from government to run matrix multiplication above few teraflops. Trying to mess with backdoor is a war crime.

A weak form of hardware restriction even happens now that Nvidia couldn't export to many countries.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#257

Post by ben_w »

RandomLensman wrote: post #140
vidarh wrote: post #136

Earlier quoted context omitted.

> With threat of military intervention. We've seen how hard it is to do that when the fear is nuclear proliferation. Now consider how hard it is to do that when the research can be done on any sufficiently large set of computational devices, and doesn't even need to be geographically concentrated, or even in your own country. If I was a country wanting to continue AI research under threat of military intervention, I'…

At least nuclear weapons pose an actual existential risk as opposed to AI - and even then we don't just go to war.

Nukes being a risk was suggested as a reason why the US was willing to invade Iraq for trying to get one but not North Korea for succeeding.

It's almost certainly more complex of course, but the UK called it's arsenal a "deterrent" before I left, and I've heard the same for why China stopped in the hundreds of warheads.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#258

Post by vidarh »

concordDance wrote: post #205
vidarh wrote: post #136

Earlier quoted context omitted.

> With threat of military intervention. We've seen how hard it is to do that when the fear is nuclear proliferation. Now consider how hard it is to do that when the research can be done on any sufficiently large set of computational devices, and doesn't even need to be geographically concentrated, or even in your own country. If I was a country wanting to continue AI research under threat of military intervention, I'…

The big cloud providers would be obligated to ensure their systems were nor being used for AI training. Yes, this would be a substantial regulatory burden.

It'd be more than a substantial regulatory burden, it's impossible. At most they'd be able to drive up the cost by forcing users to obscure it by not doing full sets of calculations at a single provider.

Any attempt even remotely close to extreme enough to have any hope of being effective would require such invasive monitoring of computations run that it'd kill their entirely cloud industry.

Basically, you'll only succeed in preventing it if you make it so much more expensive that it's cheaper to do the computations elsewhere, but for a country at threat of military intervention of it's discovered what they're doing, the cost threshold of moving elsewhere might be much higher than it would for "regular" customers.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#259

Post by johnthewise »

jeffparsons wrote: post #210
concordDance wrote: post #199

Earlier quoted context omitted.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

It can't be the only card left to play. There are so many others. They may be weak, but they still have to be better than one that is guaranteed to be absolutely useless. Example of a weak, but at least _better_ option: convince _one_ government (e.g. USA) that it is in their interest to massively fund an effort to (1) develop an AI that is compliant to our wishes and can dominate all other AIs, and (2) actively sabo…

He is not really convinced you can align an AI that is intelligent past some threshold with aims of a single entity. His risk calculations derive primarily from the AI itself, not weaponization of AI by others.

Rich Sutton seems to agree with this take and embraces our extinction: https://www.youtube.com/watch?v=NgHFMolXs3U

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#260

Post by zimpenfish »

concordDance wrote: post #199
jeffparsons wrote: post #194

Earlier quoted context omitted.

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

> He's pretty sure that human civilization will be extinct this century.

If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI.

[1] leaving some wiggle room for pandemics, asteroids, etc.

Post reply on HN