Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

521–530 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#521

Post by hodgesrm »

concordDance wrote: post #199
jeffparsons wrote: post #194

Earlier quoted context omitted.

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

> He's pretty sure that human civilization will be extinct this century.

This is certainly possible, but if so extinction seems far more likely to come from nuclear warfare and/or disasters resulting from climate change. For example, it's become apparent that our food supply more brittle than many people. Losing access to fertilizers, running out of water, or disruptions in distribution netorks can lead to mass starvation by cutting the productivity we need to feed everyone.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#522

Post by ipaddr »

jebarker wrote: post #493
jefftk wrote: post #440

Earlier quoted context omitted.

Which is very worrying! That's part of why I left big tech to work on detecting novel pandemics. [1] Luckily, the people capable of killing everyone seem not to have wanted to so far. But if the bar keeps lowering, at some point it will be accessible to "humanity is a cancer" folks and I do think we'll see serious efforts to wipe out humanity. [1] https://www.jefftk.com/p/computational-approaches-to-pathoge...

Who are the "humanity is a cancer" folk? I thought this was a trope used to discredit environmentalists?

Movies and strawmen alike.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#523

Post by JoshTriplett »

RandomLensman wrote: post #401
TeMPOraL wrote: post #390

Earlier quoted context omitted.

Since we're in Yudkowsky subthread, I'll remind here that he spent some 20 odd years explaining why containment won't work, period . You can't both contain and put to work something that's smarter than you. Another reason containment won't work is, we now know we won't even try it . Look at what happened with LLMs. We've seen the same people muse at the possibility of it showing sparks of sentience, think about the d…

I find his arguments unconvincing. Humans can and have contained human intelligence, for example (look at the Khmer Rouge, for example). Also, right now there is nothing to contain. The idea of existential risk relies on a lot of stacked-up hypotheses that all could be false. I can "create" any hypothetical risk by using that technique.

> Humans can and have contained human intelligence

ASI is not human intelligence.

As a lower bound on what "superintelligence" means, consider something that 1) thinks much faster, years or centuries every second, and 2) thinks in parallel, as though millions of people are thinking centuries every second. That's not even accounting for getting qualitatively better at thinking, such as learning to reliably make the necessary brilliant insight to solve a problem.

> The idea of existential risk relies on a lot of stacked-up hypotheses that all could be false.

It really doesn't. It relies on very few hypotheses, of which multiple different subsets would lead to death. It isn't "X and Y and Z and A and B must all be true", it's more like "any of X or Y or (Z and A) or (Z and B) must be true". Instrumental convergence (https://en.wikipedia.org/wiki/Instrumental_convergence) nearly suffices by itself, for instance, but there are multiple other paths that don't require instrument convergence to be true. "Human asks a sufficiently powerful AI for sufficiently deadly information" is another whole family of paths.

(Also, you keep saying "could" while speaking as if it's impossible for these things to not to be false.)

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#524

Post by sebastianz »

TeMPOraL wrote: post #353
sebastianz wrote: post #316

Earlier quoted context omitted.

> Which is completely irrelevant to the topic at hand anyway. It's not irrelevant. I was implying that you did not specify who is doing this military intervention that you see as a solution. What are their values, who decides the rules that the rest of the world will have to follow, with what authority, and who (and how) will the policing be done.

> I was implying that you did not specify who is doing this military intervention that you see as a solution. What are their values, who decides the rules that the rest of the world will have to follow, with what authority, and who (and how) will the policing be done. Like nuclear non-proliferation treaties or various international bans on bioweapons, but more so. The idea is that humanity is racing full steam ahead…

I see. I'm perhaps leaning skeptical most of the time, but it's hard to see how an international consensus can be reached on a topic that does not have the in-your-face type danger & fear that nuclear & bioweapons do.

(It's why I'm also pessimistic on other global consensus policies - like effective climate change action.)

I would be happy to be wrong on both counts though.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#525

Post by JoshTriplett »

ImPostingOnHN wrote: post #463
concordDance wrote: post #304

Earlier quoted context omitted.

To me it seems that the odds of us making something(s) "smarter" (better planning and modelling) than a human in the next 50 years is a coin toss. The odds of us being able to ensure its desires/goals align with what we would like are probably around 30%. And the odds of a "smarter than human" things ending up conquering or killing the humans if they desire to do so being around 90%. This naturally gives a probably o…

I would put the numbers much lower, but if your bar for legislation is ">1% risk of damage" , we've got a lot of climate change legislation that's decades overdue to pass first, and success or failure there will prove whether we can do something similar with AI.

> we've got a lot of climate change legislation that's decades overdue to pass first

We can do more than one thing at once, and we need to.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#526

Post by troupo »

RandomLensman wrote: post #433
Mordisquitos wrote: post #338

Earlier quoted context omitted.

> The latter would kind of rule out any quantum computing, too, as the only real explanation there is through maths, not words. Maths are human terms.

If the level of explanation is something that the person suffering from the decision need not necessarily understand, fair enough. But then is the limit that one human on earth can understand it? They way I understand the thrust of the regulation that isn't the intention.

> But then is the limit that one human on earth can understand it? They way I understand the thrust of the regulation that isn't the intention.

Yes, that is the intention when it comes to humans interacting with systems.

https://news.ycombinator.com/edit?id=38113944

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#527

Post by jefftk »

jebarker wrote: post #493
jefftk wrote: post #440

Earlier quoted context omitted.

Which is very worrying! That's part of why I left big tech to work on detecting novel pandemics. [1] Luckily, the people capable of killing everyone seem not to have wanted to so far. But if the bar keeps lowering, at some point it will be accessible to "humanity is a cancer" folks and I do think we'll see serious efforts to wipe out humanity. [1] https://www.jefftk.com/p/computational-approaches-to-pathoge...

Who are the "humanity is a cancer" folk? I thought this was a trope used to discredit environmentalists?

Most people think continued existence of humanity is good, which includes environmentalists. But there are some people who either think (a) avoiding suffering is overwhelmingly important, to the extent that a universe without life is ideal or (b) the world would be better off without humans because of the vast destruction we cause. I'm not claiming this is a common view, but it doesn't have to be a common view to result in destruction if destruction gets sufficiently straightforward.

For some writing in this direction, see https://longtermrisk.org/risks-of-astronomical-future-suffer... which argues that as suffering-focused people they should not try to kill everyone primarily because they are unlikely to succeed and https://philarchive.org/archive/KNUTWD ("Third, the negative utilitarian can argue that losing what currently exists on Earth would not be much of a loss, because of the following very plausible observation: overall, sentient beings on Earth fare terribly badly. The situation is not terrible for every single individual, but it is terrible when all individuals are considered. We can divide most sentient beings on Earth into the three categories of humans, non-human animals in captivity, and wild non-human animals, and deal with them in turn. ...")

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#528

Post by m3kw9 »

aldousd666 wrote: post #393

So if there is some fringe element who's willing to use AI as a terrorist implement, they'll have very little competition and they'll be very little knowledge about how to counter them if we try to outlaw open source models. I don't really believe that we can make this tool any worse than any other tool in history that's capable of killing people in some way if it's misused, but simply driving it underground won't ma…

hand waving terrorists will use AI to cause destruction. Terrorists already have enough info on the web to create plenty of terror. LLMs isn’t gonna give them a new edge and they are not magically using LLMs with AutoGPT to creat armies of “killer bots” to create “destruction”. Right now you are just repeating talking points of people who gets them from movies

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#529

Post by hm-nah »

kylebenzle wrote: post #462
meowface wrote: post #447

Earlier quoted context omitted.

He's actually suggested it may happen within 10 years. Which makes it hard to take him seriously in general. (I do personally take his overall concerns seriously, but I don't trust his timeline predictions at all, and many of his other statements.)

Can anyone enumerate what exactly your concerns are cause I don't get it. AI I'd a great spell checker, a REALLY good spell checker, and people are losing their minds over it, I don't get it.

AI is, or will very soon be, superior to all human knowledge workers. In the spirit of capitalism, lowest cost/highest profit trumps all. With the US geriatric governance and speed of change, a lot of people will be hit by under/unemployment. All this termult happening, meanwhile the AI arms race is going on. Nation states hacking eachother using AI. Then you have the AI itself trying to escape the box…want a preview of this? Listen to “Agency” audiobook on Audible. I think it’s by Stephenson or Gibson. It’s surely going to be a wild decade before us.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#530

Post by edgyquant »

concordDance wrote: post #199
jeffparsons wrote: post #194

Earlier quoted context omitted.

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

Why exactly should I care what this person thinks?
Post reply on HN