Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

491–500 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#491

Post by oceanplexian »

digdugdirk wrote: post #443
lettergram wrote: post #425

Earlier quoted context omitted.

zero chance climate change (1-2 degree increase) would end humanity. 5-10 probably wouldn’t end humanity lol Nuclear war would probably kill 80-90% but even then wouldn’t kill humanity. Projections I’ve seen are only like 40-50% of the countries hit. AI is scary if they tie it into everything and people slowly stop being capable. Then something happens, and we can’t boil water. Beyond that scenario I don’t see the ri…

That's an incredibly confident statement that very much misses the point. 50% loss of the global human population means we don't come back to the current level of technological development. 80-90% means we go back to the stone age. Photosynthesis stops working at temperatures very close to our current high temps. Pollen dies at temperatures well within those limits - we're just lucky we haven't had massive heatwaves…

The Earth was full of plants 65 million years ago when the atmosphere had like 4000PPM of CO2. most of those plants are still around.

The people saying plants will stop photosynthesis are taking you for a ride. Climate Change might certainly have some negative effects but “plants won’t exist” is not one of them.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#492

Post by ahamm »

kylebenzle wrote: post #462
meowface wrote: post #447

Earlier quoted context omitted.

He's actually suggested it may happen within 10 years. Which makes it hard to take him seriously in general. (I do personally take his overall concerns seriously, but I don't trust his timeline predictions at all, and many of his other statements.)

Can anyone enumerate what exactly your concerns are cause I don't get it. AI I'd a great spell checker, a REALLY good spell checker, and people are losing their minds over it, I don't get it.

I used to think that this fear was driven by rational minds until I read Michael Lewis' "Going Infinite" and learned more about effective altruism [EA].

Michael Lewis writes that what is remarkable about these guys [i.e. EAs] is that they're willing to follow their beliefs to their logical conclusion (paraphrasing) without regard to social cost/consequences or just inconvenience of it all. In fact, in my estimation this is just the definition of religious fundamentalism, and gives us a new lens with which to understand EA and the well funded brain children of the movement.

Every effective religion needs a doomsday scenario, or some 'second coming' apocalyptic like scenario (not sure why, just seems to be a pattern). I think all this fear mongering around AI is just that - it's based on irrational belief at the end of the day - its humanism rewashed into tech 'rationalism' (which was originally just washed from Christianity et. al.)

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#493

Post by jebarker »

jefftk wrote: post #440
itsoktocry wrote: post #423

Earlier quoted context omitted.

> how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

Which is very worrying! That's part of why I left big tech to work on detecting novel pandemics. [1] Luckily, the people capable of killing everyone seem not to have wanted to so far. But if the bar keeps lowering, at some point it will be accessible to "humanity is a cancer" folks and I do think we'll see serious efforts to wipe out humanity. [1] https://www.jefftk.com/p/computational-approaches-to-pathoge...

Who are the "humanity is a cancer" folk? I thought this was a trope used to discredit environmentalists?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#494

Post by bumby »

itsoktocry wrote: post #423
jefftk wrote: post #325

Earlier quoted context omitted.

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

> how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

How does that number compare to the number of people (experts and laypeople) who have access to the mentioned AI models?

Risk = severity x likelihood

I think the OP's point was that AI increases the likelihood by dramatically increasing the access to that level of knowledge.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#495

Post by sgu999 »

logicchains wrote: post #399
sgu999 wrote: post #191

Earlier quoted context omitted.

We don't have a fully globalised culture yet, what an average American wants isn't what an average Japanese wants. Of course even then, there are disparities and outliers, but we can't really overlook that if you "want" an SUV or an Apple Watch, that's quite likely going to be because the people around you find it valuable. I'd go even a bit further as to say that what is important is probably what people "need", not…

>I'd go even a bit further as to say that what is important is probably what people "need", not "want". Which makes you a self-righteous prick, thinking you have the right to determine what other people should and shouldn't have.

Did I prescribe anything in that regard?

What does it make you, to think that you are entitled to anything you want regardless of resources limits and collective good?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#496

Post by sokoloff »

tensor wrote: post #482
concordDance wrote: post #199

Earlier quoted context omitted.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

If humans go extinct it sure as hell won't be AI that does it, but rather good old human ignorance and aggression.

I think the probability of humans eventually going extinct is 1. Avoiding it requires almost unimaginable levels of cooperation and technical advances.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#497

Post by JoshTriplett »

itsoktocry wrote: post #423
jefftk wrote: post #325

Earlier quoted context omitted.

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

> how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

There is a massive difference between "there are a few thousand people in the world who might have the specialized knowledge to kill billions" and "anyone with a handful of GPUs or access to a handful of cloud systems could have a conversation with LatestAIUnlimited-900T and make a few specialized mail orders".

The Venn diagram of "people who could arrange to kill most of humanity without being stopped" and "people who want to kill most of humanity" is thankfully the empty set. If the former set expands to millions of people, that may not remain true.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#498

Post by RandomLensman »

troupo wrote: post #484
RandomLensman wrote: post #160

Earlier quoted context omitted.

You can have ex-post narratives for what humans do, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issue of appeals processes is totally separate. That doesn't mean we handover everything to AI, but the idea that we understand human decisions cannot be the reason for that. For some high risk things like controlling certain complex machinery or processes AI migh…

> You can have ex-post narratives for what humans did, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issue of appeals processes is totally separate. Demagoguery. Already right now you have police accusing innocent people because some ML system told them so, and China running the world' largest social credit system based on nothing. > For some high risk things…

You want a correct explanation, not some made up ex-post stuff.

Incorrect. High-risk systems indeed include certain machinery and processes, for example, in transport or in medical devices as well as operating critical infrastructure (https://www.europarl.europa.eu/news/en/headlines/society/202...).

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#499

Post by jacquesm »

jcims wrote: post #455
jacquesm wrote: post #363

Earlier quoted context omitted.

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

>The stories from that night continue to haunt me today. These conversations are oddly precious and intimate. It's so difficult to find someone that is willing to even 'go there' with you let alone someone that is capable of creatively and fearlessly advancing the conversation.

Let's just say I'm happy they're my friends and not my enemies :)

It's pretty sobering to realize how intellect applied to bad stuff can lead to advancing the 'state of the art' relatively quickly once you drop the usual constraints of ethics and morality.

To make a Nazi parallel: someone had to design the gas chambers, someone had to convince themselves that this was all ok and then go home to their wives and kids to be a loving father again. That sort of mental compartmentalization is apparently what humans are capable of and if there is any trait that we have that frightens me then that's the one. Because it allows us to do the most horrible things imaginable because we simply can imagine them. There are almost no restraints on actual execution given the capabilities themselves and it need not be very high tech to be terrible and devastating in effect.

Technology acts as a force multiplier though, so once you take a certain concept and optimize it using technology suddenly single unhinged individuals can do much more damage than they could ever do in the past. That's the problem with tools: they are always dual use and once tools become sufficiently powerful to allow a single individual to create something very impressive they likely allow a single individual to destroy something 100, 1000 or a million times larger than that. This asymmetry is limited only by our reach and the power of the tools.

You can witness this IRL every day when some hacker wrecks a company or one or more lives on the other side of the world. Without technology that kind of reach would be impossible.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#500

Post by marvin »

kylebenzle wrote: post #462
meowface wrote: post #447

Earlier quoted context omitted.

He's actually suggested it may happen within 10 years. Which makes it hard to take him seriously in general. (I do personally take his overall concerns seriously, but I don't trust his timeline predictions at all, and many of his other statements.)

Can anyone enumerate what exactly your concerns are cause I don't get it. AI I'd a great spell checker, a REALLY good spell checker, and people are losing their minds over it, I don't get it.

Humans are likely far less intelligent/capable than systems that will soon be practically feasible to engineer, and we will lose to a generally superhuman AI that has sufficiently different objectives than we do. So an accident with a runaway a system like that would be catastrophic.

Then Yudkowsky spins a gauntlet of no-evidence hypotheses for why such an accident is inevitable and leads to the death of literally all humans in the same instant.

But the first part of the argument is something that will be the critical piece of the engineering of such systems.

Post reply on HN