Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

401–410 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#401

Post by RandomLensman »

TeMPOraL wrote: post #390
RandomLensman wrote: post #317

Earlier quoted context omitted.

I don't even think we need alignment, just containment. The probabilities are really just too shaky for me to estimate. Not sure I would put a high probability of intelligent thing being dangerous by itself, for example.

Since we're in Yudkowsky subthread, I'll remind here that he spent some 20 odd years explaining why containment won't work, period . You can't both contain and put to work something that's smarter than you. Another reason containment won't work is, we now know we won't even try it . Look at what happened with LLMs. We've seen the same people muse at the possibility of it showing sparks of sentience, think about the d…

I find his arguments unconvincing. Humans can and have contained human intelligence, for example (look at the Khmer Rouge, for example).

Also, right now there is nothing to contain. The idea of existential risk relies on a lot of stacked-up hypotheses that all could be false. I can "create" any hypothetical risk by using that technique.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#402

Post by logicchains »

concordDance wrote: post #221
logicchains wrote: post #150

Earlier quoted context omitted.

LessWrongers seem to fear a superintelligence that was 95%+ effective at persuading/manipulating humans to achieve whatever its ends were. But this would require the AI being able to predict the future to a degree that wouldn't be remotely possible given the computational power available to it, due to exponentially increasing difficulty.

I'm confused. We know persuasive humans exist, we employ them as politicians or sales people. There are also many unpersuasive humans. Given we seem to have a decent range of persuasiveness even amongst these very very similar minds, why do you think the upper limit for persuasiveness is a charismatic human? Though even if that WAS the limit I'd still be somewhat worried due to the possibility of that persuasiveness…

>Given we seem to have a decent range of persuasiveness even amongst these very very similar minds, why do you think the upper limit for persuasiveness is a charismatic human

Because there's a hard limit on how much people can be made to act against their own self interest.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#403

Post by TeMPOraL »

pjc50 wrote: post #340
timmytokyo wrote: post #187

Earlier quoted context omitted.

His solution -- assuming it's the same as the one proposed by Yudkowsky, who he quotes -- is launching air strikes against data centers hosting AI computation. So yeah, imaginary.

Ah, so we're going to start the global nuclear exchange with China based on the paranoid fantasies of some guy on Twitter? And this is supposed to be taken seriously?

Well, you're reacting to a HN user paraphrasing what seems to be a single sentence taken out of context of the actual argument. Did you expect it to make sense?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#404

Post by vidarh »

TeMPOraL wrote: post #355
vidarh wrote: post #136

Earlier quoted context omitted.

> With threat of military intervention. We've seen how hard it is to do that when the fear is nuclear proliferation. Now consider how hard it is to do that when the research can be done on any sufficiently large set of computational devices, and doesn't even need to be geographically concentrated, or even in your own country. If I was a country wanting to continue AI research under threat of military intervention, I'…

Yes, it's as hard as you describe. But it seems that there are no easier options available - so this is the least hard thing we can do to mitigate the AI x-risk right now.

How do you intend to prevent people from carrying out general purpose computation?

Because that is what you'd need to do. You'd need to prevent the availability of any device where users can either directly run software that has not been reviewed, or that can be cheaply enough stripped of CPU's or GPU's that are capable of running un-reviewed software.

That review would need to include reviewing all software for "computational back doors". Given how often we accidentally create Turing-complete mechanisms in games or file formats where it was never the intent, preventing people from intentionally trying to sneak past a way of doing computations is a losing proposition.

There is no way of achieving this compatible with any resembling a free society.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#405

Post by RandomLensman »

TeMPOraL wrote: post #368
RandomLensman wrote: post #134

Earlier quoted context omitted.

No-one wants a nuclear war over imaginary risks, so not happening.

We better find a way to make it happen, because right now, climate change x-risk mitigation is also not happening, for the same reason.

Climate change mitigation is actually happening. Huge investment flows are being redirected and things are being retooled, you might not see it, but don't get caught out by the doomers there.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#406

Post by logicchains »

strken wrote: post #322
logicchains wrote: post #150

Earlier quoted context omitted.

LessWrongers seem to fear a superintelligence that was 95%+ effective at persuading/manipulating humans to achieve whatever its ends were. But this would require the AI being able to predict the future to a degree that wouldn't be remotely possible given the computational power available to it, due to exponentially increasing difficulty.

Why would you need to predict the future to be convincing? Charismatic people aren't acting like oracles, they're just...being charming. They frame actions in ways that make them look good, do little favours for you, stroke your ego, and otherwise have a good grasp of human social dynamics. Sure, there's an aspect of understanding the other person, but chaos theory doesn't stop politicians from obtaining and keeping…

>Why would you need to predict the future to be convincing?

You need to be able to predict how people will react to your words.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#407

Post by Teever »

zarzavat wrote: post #374
jacquesm wrote: post #363

Earlier quoted context omitted.

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

A real world taste of what happens when smart terrorists decide to attack: https://en.m.wikipedia.org/wiki/Tokyo_subway_sarin_attack Chemical weapons are absolutely terrifying , especially the modern ones like VX. In recent years they have mostly been used for targeted assassinations by state actors (Kim Jong Nam, Salisbury Novichok poisonings). If AI makes this stuff easier to carry out then we are completely fucked…

Imagine an attack with mass produced prions that are desseminated in the food chain.

By the time people start to realize what's going on it will be too late.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#408

Post by logicchains »

FeepingCreature wrote: post #164
logicchains wrote: post #150

Earlier quoted context omitted.

LessWrongers seem to fear a superintelligence that was 95%+ effective at persuading/manipulating humans to achieve whatever its ends were. But this would require the AI being able to predict the future to a degree that wouldn't be remotely possible given the computational power available to it, due to exponentially increasing difficulty.

This is true if and only if human intelligence is anywhere close to any theoretical maximums. I propose an alternate hypothesis: human intelligence is weak and easy to exploit. The only reason it doesn't happen more (and it already happens a lot!) is that we're too stupid to do it reliably. Consider the amount of compute needed to beat the strongest chess grandmaster that humanity has ever produced, pretty much 100%…

>This is true if and only if human intelligence is anywhere close to any theoretical maximums

Humans are capable of logical reasoning from first principles. You can fool some of the people all of the time, but no words are sufficient to convince people capable of reasoning to do things that are clearly not in their own self interest.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#409

Post by veidr »

"AI Godfather" is... pretty fucking ludicrous. Prominent figure, notable researcher? Sure.

But it's like calling Neil deGrasse Tyson (who I enjoy a lot; Astrophysics for People In A Hurry was great) the "Godfather of Physics".

Both "there's not one" and "anyway, if there were, he wouldn't be it" apply.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#410

Post by logicchains »

arisAlexis wrote: post #236

There were 3 Turing winners. The majority 2/3 Hinton and Bengio have been very outspoken about catastrophic risks to humanity. The minority 1 Lecunn who happens to be the only one on fat payroll from FB and with a vested interest goes public every day proclaiming humanity should trust his judgment. Let's bet humanity on this one person?

Bengio only won a Turing award for ideas he stole from Schmidhuber, and Schmidhuber is very anti-regulation.
Post reply on HN