Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

651–660 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#651

Post by TeMPOraL »

laverya wrote: post #429
itsoktocry wrote: post #423

Earlier quoted context omitted.

> how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

And how many of them are radical religious nutjobs? I think it is plausible that "can spend decades learning a scientific craft" and "wants to kill everyone on the planet" are mutually exclusive personality traits.

Wasn't there a study that showed there are disproportionally many engineers among the more violent "radical religious nutjobs"? The hypothesis I heard for this was that STEM people are used to complex arguments and more abstract reasoning, so while they may be harder to initially convince, when they do get convinced, they're more likely to follow the new beliefs to their logical conclusions...

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#652

Post by 4bpp »

oceanplexian wrote: post #491
digdugdirk wrote: post #443

Earlier quoted context omitted.

That's an incredibly confident statement that very much misses the point. 50% loss of the global human population means we don't come back to the current level of technological development. 80-90% means we go back to the stone age. Photosynthesis stops working at temperatures very close to our current high temps. Pollen dies at temperatures well within those limits - we're just lucky we haven't had massive heatwaves…

The Earth was full of plants 65 million years ago when the atmosphere had like 4000PPM of CO2. most of those plants are still around. The people saying plants will stop photosynthesis are taking you for a ride. Climate Change might certainly have some negative effects but “plants won’t exist” is not one of them.

It's the "arguments as soldiers" pattern. The idea that global warming will make plants incapable of photosynthesis helps the cause of fighting against global warming. You wouldn't betray the cause by refuting the idea ("stabbing your own soldiers in the back"), would you?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#653

Post by Terretta »

tensor wrote: post #482
concordDance wrote: post #199

Earlier quoted context omitted.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

If humans go extinct it sure as hell won't be AI that does it, but rather good old human ignorance and aggression.

> If humans go extinct it sure as hell won't be AI that does it, but rather good old human ignorance and aggression.

And the shift to kakistocracy.

https://en.wikipedia.org/wiki/Kakistocracy

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#654

Post by pixl97 »

jebarker wrote: post #539
jefftk wrote: post #527

Earlier quoted context omitted.

Most people think continued existence of humanity is good, which includes environmentalists. But there are some people who either think (a) avoiding suffering is overwhelmingly important, to the extent that a universe without life is ideal or (b) the world would be better off without humans because of the vast destruction we cause. I'm not claiming this is a common view, but it doesn't have to be a common view to res…

This doesn't really answer my question. These are arguments for why threat actors might hold the belief that "humanity is cancer", but it doesn't actually provide evidence that this is a credible threat in the real world.

The calculation of considering something is a credible threat directly corresponds with the ease of implementation.

If I said "I am going to drop a nuke on jebarker" that's not a credible threat unless I happen to be a nation state.

Now, if I said "I'm going to come to jebarkers house and blast him with a 12 gauge" and I'm from the US, that is a credible threat in most cases due to the ease of which I can get a weapon here.

And this comes down to the point of having a magic machine in which you could ask for the end of the world. The more powerful that machine gets the bigger risk it is for everyone. You ever see those people that snap and start killing everyone around them? Would you want them to have a nuke in that mood? Or a 'lets turn humanity in to grey goo' option?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#655

Post by TeMPOraL »

tppiotrowski wrote: post #420
jacquesm wrote: post #363

Earlier quoted context omitted.

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

The people who can do doomsday level harm need to complete a level of education that makes them smart enough and gives them time to consider the implications of their actions. They have a certain level of maturity before they can strike and this maturity makes them understand the gravity of their actions. Also, by this point they are probably set financially due to their education and not upset with the world (possib…

> Theorem: by the time someone reaches the intelligence level required to annihilate the world they can comprehend the implications of their actions.

That may or may not be somewhat the case in humans (there definitely are exceptions). Still, the opposite theorem, known as "Orthogonality thesis", states that, in general case, intelligence and value systems are mutually independent. There are good arguments for that being the case.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#656

Post by danenania »

randomdata wrote: post #648
danenania wrote: post #644

Earlier quoted context omitted.

That's like saying that you can regulate nuclear materials without input from nuclear physicists or regulate high risk virology without input from virologists. The goal is (or should be) to determine how we can get the benefits of the new technology (nuclear energy, vaccines, AI productivity boom) while minimizing civilizational risk (nuclear war/terrorism, bioweapons/man-made pandemics, anti-human AI applications).…

Experts are not lawmakers. We aren't looking for them to craft regulation, we're looking to hear from them about the various realistic scenarios that could play out. But we're not hearing anything. ...probably because there isn't much to hear. Like, Hinton's big warning is that AI will be used to steal identities. What does that tell us? We already know that identity isn't reliable. We've known that for centuries. Re…

I guess my experience is different. I've heard plenty about realistic scenarios. It's out there if you look for it, or even if you just spend time thinking it through. Identity theft is far from the biggest danger even with current capabilities.

Though to your point, I think part of the issue is that people who study this stuff are often hesitant to give too much detail in public because they don't want to give ideas to potentially nefarious actors before any protections are in place.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#657

Post by ben_w »

TeMPOraL wrote: post #630
bnralt wrote: post #561

Earlier quoted context omitted.

Every time I see Yudkowsky’s writing I wonder why people take him seriously. Just a couple of months back he was duped by a fake story about an airforce AI deciding to kill it’s operators in a simulation. It turned out the whole thing was thought experiment. Asking Yudkowsky what we should do about AI is like asking Green Peace what we should do about nuclear power.

If I had a cent for every time I see a HN semi-regular (i.e. not a throwaway account) falling for some rather transparent bullshit, I'd enjoy my retirement on a significant "FU money" cache... Everyone gets duped sometimes, often publicly. I remember skimming a summary of the story itself. Sounded like textbook failure mode of reinforcement learning. I was actually saddened to later learn it apparently was just a tho…

Add an imaginary cent from me, if you've not already. This had me fooled for years: https://users.math.yale.edu/public_html/People/frame/Fractal...

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#658

Post by randomdata »

danenania wrote: post #656
randomdata wrote: post #648

Earlier quoted context omitted.

Experts are not lawmakers. We aren't looking for them to craft regulation, we're looking to hear from them about the various realistic scenarios that could play out. But we're not hearing anything. ...probably because there isn't much to hear. Like, Hinton's big warning is that AI will be used to steal identities. What does that tell us? We already know that identity isn't reliable. We've known that for centuries. Re…

I guess my experience is different. I've heard plenty about realistic scenarios. It's out there if you look for it, or even if you just spend time thinking it through. Identity theft is far from the biggest danger even with current capabilities. Though to your point, I think part of the issue is that people who study this stuff are often hesitant to give too much detail in public because they don't want to give ideas…

> I've heard plenty about realistic scenarios.

Of course. Everyone has an opinion. Some of those opinions will end up be quite realistic, even if just by random chance. You don't have to be an expert to come up with right ideas sometimes.

Hinton's vision of AI being used to steal identities is quite realistic. But that doesn't make him an expert. His opinion carries no more weight than any other random hobo on the street.

> I think part of the issue is that people who study this stuff are often hesitant to give too much detail in public because they don't want to give ideas to potentially nefarious actors

Is there no realistic scenario where the outcome is positive? Surely they could speak to that, at least. What if, say, AI progressed us to post-scarcity? Many apparent experts believe post-scarcity will lead us away from a lot of the nefarious activity you speak of.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#659

Post by _cjse »

upupupandaway wrote: post #182
JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

Well, you can police the entire world quite cheaply on this or any other scientific research program by using fine-insured bounties to do it. Whether it's a good idea is a different question. https://web.archive.org/web/20220709091749/https://virtual-i...
Post reply on HN