Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

691–700 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#691

Post by breuleux »

RandomLensman wrote: post #262
concordDance wrote: post #250

Earlier quoted context omitted.

What's your view on where the X risk idea breaks down? Is it that it's impossible to make something smarter than a human? Or that such a thing wouldn't have goals/plans that require resources us humans want/need? Or that such a thing wouldn't be particularly dangerously powerful?

It could break down on any of those, for example (with the first one not forever, but could take much longer than what is mainstream thinking now). A lot of existential risk involves bold hypotheses about machines being able to "solve" a lot of things humans cannot, but we don't know how much is actually just solvable by superior intelligence vs. what just doesn't work in that way. Human collective intelligence faile…

I would argue that intelligence, at the core, is about understanding and manipulating simple systems in order to achieve specific goals. This becomes exponentially difficult with complexity (or level of chaos), so past a certain threshold, systems become essentially unintelligible. A large part of our effectiveness is in strategically simplifying our environment so that it is amenable to intelligent manipulation.

Most of the big problems we have are in situations where we need to preserve complex aspects of reality that make the systems too chaotic to properly predict, so I suspect AI won't be able to do much better regardless of how smart it is. The ability to carry out destructive brute force actions worries me more than intelligence.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#693

Post by carbotaniuman »

vidarh wrote: post #434
TeMPOraL wrote: post #392

Earlier quoted context omitted.

It all sounds very bad, but the truth is, we're going to get there anyway - in fact, we're half-way there, thanks to the influence of media conglomerates pushing DRM, adtech living off surveillance economy, all the other major tech vendors seeking control of end-user devices, and security industry that provides them all with means to achieve their goals. Somehow, no one is worried about the economy. Perhaps because i…

We're nowhere near there . Even on the most locked down devices we have, getting computation past the gatekeepers doesn't require reviews remotely detailed enough to catch attempts at sneaking these kinds of computation past, or even making them expensive, and at no point in history have even individuals been able to rent as much general purpose computational capacity for as little money. Even almost all the most loc…

There's also the bomb every datacenter doing AI thing that LeCun has proposed, which is a lot more practical than this. Or perhaps a self replicating work that destroys AIs.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#694

Post by bondarchuk »

daveguy wrote: post #671
bondarchuk wrote: post #632

Earlier quoted context omitted.

> Every time I see Yudkowsky’s writing I wonder why people take him seriously. Because he identified the potential risk of superhuman AI 20 years before almost everyone else. Sure, science fiction also identified that risk (as people here seem eager to point out), but he identified it as an actual real-world risk and has been trying to take real-world action on it for 20 years. No matter what else you think about him…

A poorly premised fantasy fiction horror story about what might happen during the interaction between a super intelligence and its creator makes for a great story, but it's not a blueprint for the future, much less a pressing concern. Misuse of current AI is far more concerning than anything dreamt up about AGI.

Yeah, for me EY's stories about an airgapped AI convincing their keepers to release it (or that other one about an AI destroying all humans because it's the most efficient way to make a gazillion paperclips) also miss the point. It's mostly the just the realization that superhuman AI might be dangerous through some kind of runaway feedback, as a very general idea, that I respect about EY's work.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#695

Post by hollerith »

kylebenzle wrote: post #461
hollerith wrote: post #446

Earlier quoted context omitted.

Yudkowsky does not expect a stop to the development of AI globally. He, and thousands of others (including me) are merely saying that if there is not a stop, then we are all dead. Yudkowsky thinks we are all dead with probability .999. I don't know the subject nearly as well as he does, which makes it impossible for me to have much certainty, so my probability is only .9. Also, it is only the development of ever-more…

I've been in data science only a few years but I fail to understand what people's issue with AI is. Best 8 can tell is it's ignorance coupled with fear mongering.

>I fail to understand what people's issue with AI is

But you haven't put much effort into trying to understand, have you?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#696

Post by bbor »

gmd63 wrote: post #663
Workaccount2 wrote: post #474

Earlier quoted context omitted.

I fully recognize the danger AI presents, and believe that it will mostly likely end up being terrible for humanity. But thanks to my own internal analysis ability and the anonymity of the internet, I am also willing to speak candidly. And I think I speak for many people in the tech community, whether they realize it or not. So here we go: My objective judgement of the situation is heavily adulterated by my incredibl…

> how can you spend your life working to get to the garden of Eden, and then not eat the fruit? Even just a taste. Because it's insect-brain level behavior to surrender in faith to some abstract achievement without understanding its viability or actual desirability.

They understand the desirability - they’ve been thinking about it for decades. AGI would be massive unexpected help in alleviating poverty, exploring the stars, advancing science, expanding access to quality medical & educational resources, etc etc etc. All of this on top of the more fundamental “IT LIVES!!” primal feeling of accomplishment and power, which I think isn’t something we can dismiss as irrational/“insect-level” offhand. What makes that feeling less virtuous than human yearning for exploration, expression, etc?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#697

Post by ac29 »

rolisz wrote: post #77
JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

And... How do you do that? Seriously, people have been talking about stoping CO2 emissions for what, 50 years? We've barely made progress there.

I think you are really underestimating how much progress has been made with renewable energy. Taking a quick look a CAISO, the California grid is currently 65% powered by renewables (mostly solar). This is a bit higher than average, but it certainly wouldnt have been nearly as high even 5 years ago, much less 25 years ago.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#698

Post by abecedarius »

api wrote: post #481
germandiago wrote: post #472

Earlier quoted context omitted.

True. We receive warnings all the time but I think what these people are afraid of is actually losing their power.

Until these same people start being concerned about DIY CRISPR kits, I do not take them seriously: https://www.the-odin.com/diy-crispr-kit/ This is dangerous in ways that require no leaps of logic or hypothetical sci-fi scenarios to imagine. I studied biology as an undergrad and can think of several insanely evil things you could do with this plus some undergrad-level knowledge of genetics and... not gonna elaborate.…

I've seen Yudkowsky and others saying that AI and biotech are both in the unusual bin of threats to all humanity, exceptions to his belief that most things are overregulated. (paraphrase of tweets from memory)

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#699

Post by bbor »

btbuildem wrote: post #448
patcon wrote: post #384

I need to resist all this regulatory resistance here, spoken amongst tech people. No, Yann. I am FULLY in support of drastic measures to mitigate and control AI research for the time being. I have no vested stake in any of the companies. I lived for a year in a building that hosted several AI events per week. I'm not ignorant This is the only territory where humanity is mounting a conversation about a REAL response t…

On one hand, we have some pie-in-the-sky, pop-culture-inspired "risk". On the other, we have the ugliness of human nature, demonstrated repeatedly countless times over millennia. These are people with immense resources, hell-bent on acquiring even more riches and power. Like always in history, every single time. They're manipulating the discourse to distract from the real present danger (them) and shift the focus to…

You’re biased towards post-cold-war fiction, IMO. I’m a cultural Luddite so have no sources, but I imagine there’s lots of stories where the Russians/Chinese/Germans/Triad/Mafia/Illuminati/Masons/etc did it, aka some creepy unseen “other” that is watching us and waiting to strike. I think a lot of people are taking this same view today — perhaps with more PC language

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#700

Post by breuleux »

hackinthebochs wrote: post #679
breuleux wrote: post #667

Earlier quoted context omitted.

I would say you are also overconfident in your own statements. > Individual humans are limited by biology, an AGI will not be similarly limited. On the other hand, individual humans are not limited by silicon and global supply chains, nor bottlenecked by robotics. The perceived superiority of computer hardware on organic brains has never been conclusively demonstrated: it is plausible that in the areas that brains ha…

Yes, every point in favor of the possibility of AGI comes with an asterisk. That's not all that interesting. We need to be competent at reasoning under uncertainty, something few people seem to be capable of. When the utility of a runaway AGI is infinitely negative, while the possibility of that outcome is substantially non-zero, rationality demands we act to prevent that outcome. >How would they force perfect alignm…

> Lack of independence implies low coordination costs

Perhaps, but it's not obvious. Lack of independence implies more back-and-forth communication with the central coordinator, whereas independent agents could do more work before communication is required. It's a tradeoff.

> the features of an artificial intelligence implies the ability to maximally utilize the abilities of the sub-components

Does it? Can you elaborate?

> It stands to reason that the more powerful the mini-brains, along with the efficiency of coordination, implies a much more capable unified intelligence.

It also implies an easier alignment problem. If an intelligence can coordinate "mini-brains" fully reliably (a big if, by the way), presumably I can do something similar with a Python script or narrow AI. Decoupling capability from independence is ideal with respect to alignment, so I'm a bit less worried, if this is how it's going to work.

Post reply on HN