Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

461–470 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#461

Post by kylebenzle »

hollerith wrote: post #446
upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

Yudkowsky does not expect a stop to the development of AI globally. He, and thousands of others (including me) are merely saying that if there is not a stop, then we are all dead. Yudkowsky thinks we are all dead with probability .999. I don't know the subject nearly as well as he does, which makes it impossible for me to have much certainty, so my probability is only .9. Also, it is only the development of ever-more…

I've been in data science only a few years but I fail to understand what people's issue with AI is. Best 8 can tell is it's ignorance coupled with fear mongering.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#462

Post by kylebenzle »

meowface wrote: post #447
concordDance wrote: post #199

Earlier quoted context omitted.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

He's actually suggested it may happen within 10 years. Which makes it hard to take him seriously in general. (I do personally take his overall concerns seriously, but I don't trust his timeline predictions at all, and many of his other statements.)

Can anyone enumerate what exactly your concerns are cause I don't get it.

AI I'd a great spell checker, a REALLY good spell checker, and people are losing their minds over it, I don't get it.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#463

Post by ImPostingOnHN »

concordDance wrote: post #304
RandomLensman wrote: post #281

Earlier quoted context omitted.

I meant could also that there are other things that could also make the risk not materialise. I'd think they are quite independent. Not sure I have any odds on those individually, other than I consider the overall risk really really low. The way I see it, there are a few things working against the risk from pure intelligence to start with (as it is with humans btw., the Nazis were not all intellectual giants, for exa…

To me it seems that the odds of us making something(s) "smarter" (better planning and modelling) than a human in the next 50 years is a coin toss. The odds of us being able to ensure its desires/goals align with what we would like are probably around 30%. And the odds of a "smarter than human" things ending up conquering or killing the humans if they desire to do so being around 90%. This naturally gives a probably o…

I would put the numbers much lower, but if your bar for legislation is ">1% risk of damage", we've got a lot of climate change legislation that's decades overdue to pass first, and success or failure there will prove whether we can do something similar with AI.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#464

Post by JackFr »

jacquesm wrote: post #363
api wrote: post #349

Earlier quoted context omitted.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

> Case in point: the Kennedy assassination changed the world

Really? In what substantive way?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#465

Post by wetmore »

tppiotrowski wrote: post #420
jacquesm wrote: post #363

Earlier quoted context omitted.

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

The people who can do doomsday level harm need to complete a level of education that makes them smart enough and gives them time to consider the implications of their actions. They have a certain level of maturity before they can strike and this maturity makes them understand the gravity of their actions. Also, by this point they are probably set financially due to their education and not upset with the world (possib…

Counterpoint: the Unabomber

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#466

Post by jncfhnb »

itsoktocry wrote: post #421
zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

> If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. If we go extinct in the next 100 years, it's not going to be from climate change. How would that even work?

Human extinction is not on the table. Societal collapse could be.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#467

Post by shafyy »

TeMPOraL wrote: post #327
concordDance wrote: post #308

Earlier quoted context omitted.

Artifical Super Intelligence. Could make humans extinct just like humans could make chimpanzees extinct. Using superior intellect to better leverage available resources than the things it/they are competing with until it out expands/outcompetes them.

Note that ways ASI can make us extinct includes all the ways we could do it to ourselves. One of the possible scenario is tricking humans into starting WWIII, perhaps entirely by accident. Another is that even being benignly applied to optimize economic activity in broad terms might have the AI strengthen the very failure modes that accelerate climate change. Point being, all the current global risks won't disappear…

> One of the possible scenario is tricking humans into starting WWIII

This sounds more like a plot of a third-grade movie than something that could happen in reality.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#468

Post by m_a_g »

JoshTriplett wrote: post #52
lazzlazzlazz wrote: post #35

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

Could you explain to me how AGI is an existential threat to humanity? All the arguments I've read so far feel so far-fetched it's unbelievable. It's in the order of making an AGI and making it control the Pentagon (!?).

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#469

Post by ImPostingOnHN »

TeMPOraL wrote: post #321
rolisz wrote: post #285

Earlier quoted context omitted.

So you're proposing replacing a hypothetical risk (Geoffrey Hinton puts the probability at 1%) with guaranteed pain and destruction, which might escalate to nuclear war? Thank you, but no thank you.

No, I'm proposing replacing a high risk scenario (I disagree with Hinton here) with a low risk scenario, which has low chance to escalate to any serious fighting. It seems people have missed what the proposed alternative is, so let me spell it out: it's about getting governments to SIGN AN INTERNATIONAL TREATY. It is NOT about having US (or anyone else) policing the rest of the world. It's not fundamentally more dang…

> No, I'm proposing replacing a high risk scenario (I disagree with Hinton here)

I disagree with you here, I think the risk is low, not high.

> it's about getting governments to SIGN AN INTERNATIONAL TREATY. It is NOT about having US (or anyone else) policing the rest of the world

We haven't been able to do that for climate change. When we do, then I'll be convinced enough that it would be feasible for AI. Until then, show me this coordination for the damage that's already happening (climate change).

> This level of coordination may seem near-impossible to achieve now, but then it's much more possible than surviving an accidental creation of an AGI.

I think the coordination required is much less possible than a scenario where we need to "survive" some sort of danger from the creation of an AGI. But we can find out for sure with climate change as an example. Let's see the global coordination. Have we solved that actual problem, yet?

Remember, everyone has to first agree to this "bomb AI" to avoid war. Otherwise, bombing/war starts. The equivalent for climate change would be bombing carbon producers. I don't see either agreement happening globally.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#470

Post by api »

patcon wrote: post #384

I need to resist all this regulatory resistance here, spoken amongst tech people. No, Yann. I am FULLY in support of drastic measures to mitigate and control AI research for the time being. I have no vested stake in any of the companies. I lived for a year in a building that hosted several AI events per week. I'm not ignorant This is the only territory where humanity is mounting a conversation about a REAL response t…

The most likely outcome of this exercise in pumping the brakes is the monopolization of high-end AI by a small number of very rich or powerful people. They will then be able to leverage it as an intelligence amplifier for themselves while renting out access to inferior versions of its capabilities to the rest of humanity and becoming even more extraordinarily wealthy.

If AI really is going to take off as much as some suspect it will, then you need to wrap your head around the sheer magnitude of the power grab we could be witnessing here.

The most plausible of the "foom" scenarios is one in which a tiny group of humans leverage superintelligence to become living gods. I consider "foom" very speculative, but this version is plausible because we already know humans will act like that while AI so far has no self-awareness or independence to speak of.

I don't think most of the actual scientists concerned about AI want that to happen, but that's what will happen in a regulatory regime. It is always okay for the right people to do whatever they want. Regulations will apply to the poors and the politically unconnected. AI will continue to advance. It will just be monopolized. If some kind of "foom" scenario is possible, it will still happen and will be able to start operating from a place of incredible privilege it inherits from the privileged people who created it.

There is zero chance of an actual "stop."

Post reply on HN