Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

791–800 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#791

Post by richardw »

NumberWangMan wrote: post #716
richardw wrote: post #177

Earlier quoted context omitted.

I think the two sides have such different perspectives. Some are optimistic builders and see only opportunity. Others are "safety oriented engineers" whose job and mindset is to build guarantees into systems, or secure countries from external dangers. The latter has a very hard time with the lack of guarantees with a system that is only ever going to increase in capability. Choose any limit. Any. AI will be smart but…

Zvi Mowshowitz has a nuanced viewpoint I like -- we should be very careful and move very slowly with any tech that has a chance of wiping out humanity, and embrace the "move fast and break things" attitude with everything else, e.g. make it much easier to build housing, review and scrap any licensing requirements that do more harm than good, etc.

That sounds very rational and I'd agree fully. I suppose the challenge is that the optimists think there is no chance it'll wipe out humanity so they place AI in the "move fast and break things" camp. Getting them to not be dismissive is difficult when they see no danger.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#792

Post by pixl97 »

jebarker wrote: post #705
pixl97 wrote: post #654

Earlier quoted context omitted.

The calculation of considering something is a credible threat directly corresponds with the ease of implementation. If I said "I am going to drop a nuke on jebarker" that's not a credible threat unless I happen to be a nation state. Now, if I said "I'm going to come to jebarkers house and blast him with a 12 gauge" and I'm from the US, that is a credible threat in most cases due to the ease of which I can get a weapo…

The problem with this analysis is that we have complete uncertainty about the probability that AGI/ASI will be developed in any particular time frame. Anyone that says otherwise is lying or deluded. So the risk equation is impossible to calculate for AGI/ASI wiping out humanity. And since you appear to be evaluating the damage as essentially infinite, i.e. existential risk, you're advocating that as long as there's a…

No, no need to calculate infinities. Instead lets imagine something with an IQ within one exponent higher of ours. This is exceptionally difficult to even imagine as below 70 we consider a person disabled in some form or another. Once we get in the range of IQ over 170 or so it really starts to break. When we get into multimodal self learning systems there may be vast amounts of knowledge encoded into its networks that we simply don't know the right question to ask as humans. Attempt to think of what other AIs at or near that level could ask each other.

Our different systems of intelligence keep falling back to ideas of human intelligence and its limitations. Things like we don't multitask well, our best attention is given to one thing at a time. Things like our highest bandwidth senses are our ears and eyes, and even then they are considerably bandwidth constricted. Human limits like our abilities cannot be horizontally scaled in an easily measurable way, and adding more people massively increases the networking costs of accomplishing a goal.

If for some reason the smartest AI can only reach the levels of the smartest human beings (which at least to me makes no sense), then this is still massively powerful, as of this point pesky human problems like 8 hours of sleep are not needed. If human level AI power can be shrunk down to near cellphone size and power consumption, which again doesn't seem out of the realm of physics, lays down the framework for an intelligence explosion at the rate which we can print chips out and assemble them.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#793

Post by ben_w »

RandomLensman wrote: post #751
pixl97 wrote: post #735

Earlier quoted context omitted.

We don't have an answer yet to what the form factor of AGI/ASI will be, but if it's anything like current trends the idea of 'killed' is laughable. You can be killed because your storage medium and execution medium are inseparable. Destroy the brain and you're gone, and you don't even get to copy yourself. With AGI/ASI if we can boot it up from any copy on a disk if we have the right hardware then at the end of the d…

No power and it its already dead. Destroy copies, dead. Really not complicated at all.

You're the only example I know of, of someone using "containment" to mean "extermination" in the context of AI.

Extermination might work, though only on models large enough that people aren't likely to sneak out of the office with copies for their own use and/or just upload copies to public places (didn't that already happen with the first public release of Stable Diffusion, or am I misremembering?)

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#794

Post by ben_w »

oceanplexian wrote: post #734
dghlsakjg wrote: post #642

Earlier quoted context omitted.

Hal 9000, the malevolent AI in 2001: a space Odyssey, was in one of the most respected films of all time three years before Yudkowsky was born. Plenty of people have been beating that drum for years.

Mary Shelly wrote about the same philosophical issue in 1818 when she wrote Frankenstein. My take is that AI influencers don’t read a lot of books, because if they did they’d have much more intelligent commentary.

A Frankenstein's creation scenario would be a fantastic win in AI. Extremely smart, started off kind, only turned cruel when literally every single human he met was violently repulsed by the mere sight of him.

If you want AI gone wrong, Magician's Apprentice.

If you want misalignment between what you wanted and what you actually asked for, Greek myths, Midas or Tithonus would both suffice.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#795

Post by jebarker »

pixl97 wrote: post #792
jebarker wrote: post #705

Earlier quoted context omitted.

The problem with this analysis is that we have complete uncertainty about the probability that AGI/ASI will be developed in any particular time frame. Anyone that says otherwise is lying or deluded. So the risk equation is impossible to calculate for AGI/ASI wiping out humanity. And since you appear to be evaluating the damage as essentially infinite, i.e. existential risk, you're advocating that as long as there's a…

No, no need to calculate infinities. Instead lets imagine something with an IQ within one exponent higher of ours. This is exceptionally difficult to even imagine as below 70 we consider a person disabled in some form or another. Once we get in the range of IQ over 170 or so it really starts to break. When we get into multimodal self learning systems there may be vast amounts of knowledge encoded into its networks th…

But the problem still stands that we can't estimate how likely any of this is to occur. So in the overall risk equation we have a term that is essentially completely uncertain and therefore varying that term can yield any arbitrary risk level you want.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#796

Post by Teever »

jacquesm wrote: post #411
Teever wrote: post #407

Earlier quoted context omitted.

Imagine an attack with mass produced prions that are desseminated in the food chain. By the time people start to realize what's going on it will be too late.

You could just mix it in with animal feed... don't give any militant vegetarians ideas now. Not that the meat industry would ever score an own goal like that. https://en.wikipedia.org/wiki/Bovine_spongiform_encephalopat...

A less militant vegetarian would want to induce that red meat allergy from lone star ticks.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#797

Post by ben_w »

recursive wrote: post #724
berniedurfee wrote: post #515

Earlier quoted context omitted.

I don’t think we’re close, not even within 50 years of real AGI. Which may or may not decide to wipe us out. However, even given the current state of “AI”, I think there are countless dangers and risks. The ability to fabricate audio and video that’s indistinguishable from the real thing can absolutely wreck havoc on society. Even just the transition of spam and phishing from poorly worded blasts with grammatical err…

> The ability to fabricate audio and video that’s indistinguishable from the real thing can absolutely wreck havoc on society. Only for as long people hold on to the idea that videos are reliable. Video is a medium. It will become as reliable as text. The existence of text, and specifically lies, has not wrecked society.

Lots of people have fought and died in wars over the belief that a book was the literal Word of God, and some of those wars lasted a lifetime.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#798

Post by recursive »

ben_w wrote: post #797
recursive wrote: post #724

Earlier quoted context omitted.

> The ability to fabricate audio and video that’s indistinguishable from the real thing can absolutely wreck havoc on society. Only for as long people hold on to the idea that videos are reliable. Video is a medium. It will become as reliable as text. The existence of text, and specifically lies, has not wrecked society.

Lots of people have fought and died in wars over the belief that a book was the literal Word of God, and some of those wars lasted a lifetime.

In the future we can do that with videos. But it won't be anything new.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#799

Post by ben_w »

recursive wrote: post #798
ben_w wrote: post #797

Earlier quoted context omitted.

Lots of people have fought and died in wars over the belief that a book was the literal Word of God, and some of those wars lasted a lifetime.

In the future we can do that with videos. But it won't be anything new.

I'm not disputing that, I'm saying there's precedent to expect it will wreck societies.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#800

Post by nojvek »

The executive order from Biden very much read that defense will spend billions on AI and that money will only be spent on companies that lobby heavily in their favor (Microsoft via OpenAI and Alphabet via DeepMind).

Now I love Yann, but Meta is a big player as well. They’re more open than OpenAI and DeepMind but keep plenty close to chest.

The real risk is Big Tech getting ever larger controlling what laws get passed and who is allowed to do what.

Post reply on HN