Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

711–720 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#711

Post by danenania »

randomdata wrote: post #687
danenania wrote: post #686

Earlier quoted context omitted.

Check out Dwarkesh Patel’s podcast.

What makes him an expert in the subject matter? A cursory glance suggests that his background is in CS, not anything related to social or humanitarian issues. It is not completely impossible for someone to have expertise in more than one thing, but it is unusual as there is only so much time in the day and building expertise takes a lot of time.

I don’t mean Dwarkesh himself, though he asks great questions. He’s had some very knowledgeable guests.

The most recent episode with Paul Christiano has a lot of good discussion on all these topics.

I’d suggest evaluating the arguments and ideas more on the merits rather than putting so much emphasis on authority and credentials. I get there can be value there, but no one is really an “expert” in this subject yet and anyone who claims to be probably has an angle.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#712

Post by Xirgil »

breuleux wrote: post #672
JoshTriplett wrote: post #523

Earlier quoted context omitted.

> Humans can and have contained human intelligence ASI is not human intelligence. As a lower bound on what "superintelligence" means, consider something that 1) thinks much faster, years or centuries every second, and 2) thinks in parallel, as though millions of people are thinking centuries every second. That's not even accounting for getting qualitatively better at thinking , such as learning to reliably make the n…

> As a lower bound on what "superintelligence" means, consider something that 1) thinks much faster, years or centuries every second, and 2) thinks in parallel, as though millions of people are thinking centuries every second. I'm fairly certain that what you describe is physically impossible. Organic brains may not be fully optimal, but they are not that many orders of magnitude off.

Not only is such an entity possible, it's already here, the only difference is clock speed. Corportations (really any large organization devoted to some kind of intellectual work, or you could say human society as a whole) can be thought of as a superintelligent entity composed of biological processors running in parallel. They that can store, process, and create information at superhuman speeds compared to the individual. And given that our brains evolved restricted by the calories available to our ancient ancestors, the size of the human pelvis, and a multitude of other obsolete factors, that individual processing unit clearly far from optimal. But even if you just imagine the peak demonstrated potential of the human brain in the form of an AI process it's easy to see how that could be scaled to massively superhuman levels. Imagine a corporation that could recruit from a pool of candidates (only bound by their computing power) each with the intellect of John von Neumann, with a personality fine tuned for their role, that can think faster by increasing their clock speed, that can access, learn and communicate information near instantly, happily work 24/7 with zero complaint or fatigue, etc, etc. Imagine how much more efficient (how much more superintelligent) that company would be compared to its competition.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#713

Post by nonbirithm »

bbor wrote: post #696
gmd63 wrote: post #663

Earlier quoted context omitted.

> how can you spend your life working to get to the garden of Eden, and then not eat the fruit? Even just a taste. Because it's insect-brain level behavior to surrender in faith to some abstract achievement without understanding its viability or actual desirability.

They understand the desirability - they’ve been thinking about it for decades. AGI would be massive unexpected help in alleviating poverty, exploring the stars, advancing science, expanding access to quality medical & educational resources, etc etc etc. All of this on top of the more fundamental “IT LIVES!!” primal feeling of accomplishment and power, which I think isn’t something we can dismiss as irrational/“insect…

Maybe that is the crux of the problem. Human yearning is dual-use.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#714

Post by pixl97 »

shafyy wrote: post #467
TeMPOraL wrote: post #327

Earlier quoted context omitted.

Note that ways ASI can make us extinct includes all the ways we could do it to ourselves. One of the possible scenario is tricking humans into starting WWIII, perhaps entirely by accident. Another is that even being benignly applied to optimize economic activity in broad terms might have the AI strengthen the very failure modes that accelerate climate change. Point being, all the current global risks won't disappear…

> One of the possible scenario is tricking humans into starting WWIII This sounds more like a plot of a third-grade movie than something that could happen in reality.

If I snached someone out of 1850 and presented them the world in which we live the entire premise of our existence would seem to be nonsensical fiction to them, and yet it would be real.

And using movie plots as a basis generally doesn't directly work as fiction has to make sense to sell whereas reality has no requirement of making sense to people. The reality we live in this day is mostly driven by people for people (though many would say by corporations for corporations) and therefore things still make sense to us people. When and if an intellect matches or exceeds that of humans it can easily imagine situations where 'the algorithm' does things humans don't comprehend, but because we make more (whatever) we keep giving it more power to do its job in an autonomous fashion. It is not difficult to end up in a situation where you have a system that is not well understood by anyone and you end up cargo culting it to hope it continues working into the future.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#715

Post by pixl97 »

oceanplexian wrote: post #503
hollerith wrote: post #460

Earlier quoted context omitted.

>He is not really convinced you can align an AI that is intelligent past some threshold with aims of a single entity. That's a little bit inaccurate: he believes that it is humanly possible to acquire a body of knowledge sufficient to align an AI (i.e., to aim it at basically any goal the creators decide to aim it at), but that it is extremely unlikely that any group of humans will manage to do before unaligned AI ki…

I wonder if I, too, can simply rehash the plot of Terminator 2: Judgment Day and then publish research papers on how we will use AI’s to battle it out with other AI’s.

Yes, and honestly it's a good damned idea to start now because we're already researching autonomous drones and counter autonomous drones that are autonomous themselves.

While the general plot of T2 is bullshit, the idea of autonomous weapon systems at scale should be an 'oh shit' moment for everyone.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#716

Post by NumberWangMan »

richardw wrote: post #177
brutusborn wrote: post #83

Earlier quoted context omitted.

Who do you find persuasive and what material should I read/watch to understand their POV? So far I’ve read a bunch of lesswrong posts and listened to some Eliezer talks but still can’t understand the basis of their arguments or when I do, they seem very vague. The only suggestion that makes sense to me is from the FEP crowd. Essentially if someone sets up at AI with an autopoietic mechanism then it would be able to t…

I think the two sides have such different perspectives. Some are optimistic builders and see only opportunity. Others are "safety oriented engineers" whose job and mindset is to build guarantees into systems, or secure countries from external dangers. The latter has a very hard time with the lack of guarantees with a system that is only ever going to increase in capability. Choose any limit. Any. AI will be smart but…

Zvi Mowshowitz has a nuanced viewpoint I like -- we should be very careful and move very slowly with any tech that has a chance of wiping out humanity, and embrace the "move fast and break things" attitude with everything else, e.g. make it much easier to build housing, review and scrap any licensing requirements that do more harm than good, etc.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#717

Post by kaibee »

ipaddr wrote: post #519
JoshTriplett wrote: post #497

Earlier quoted context omitted.

There is a massive difference between "there are a few thousand people in the world who might have the specialized knowledge to kill billions" and "anyone with a handful of GPUs or access to a handful of cloud systems could have a conversation with LatestAIUnlimited-900T and make a few specialized mail orders". The Venn diagram of "people who could arrange to kill most of humanity without being stopped" and "people w…

But someone could ask the AI how to stop it.

It tends to be a lot easier to destroy than create, because entropy sucks like that.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#718

Post by pixl97 »

varjag wrote: post #318
TeMPOraL wrote: post #287

Earlier quoted context omitted.

> Example of a weak, but at least _better_ option: convince _one_ government (e.g. USA) that it is in their interest to massively fund an effort to (1) develop an AI that is compliant to our wishes and can dominate all other AIs, and (2) actively sabotage competing efforts. I'd reconsider your revision of your estimate of Yudkowsky, as you seem to be dropping it for not proposing the very ideas he spent the last 20+…

Threat of war is always the final argument in international affairs, and you can tell how serious a treaty is by how directly it acknowledges that fact. Montreal Protocol have worked despite no threats of violence, on a not entirely dissimilar problem. Tho I share the skepticism on solution to alignment problem.

You mean kind of worked... global CFC emissions are on the rise again.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#719

Post by ImPostingOnHN »

4bpp wrote: post #670
ImPostingOnHN wrote: post #633

Earlier quoted context omitted.

to continue contributing to the harm caused by AI only requires that 1 person use an existing model running off their laptop to foment real-world violence or spread disinformation en masse on social media, or use it on a cheap swarm of weaponized/suicide drones, for a few examples

There's a qualitative barrier there. The AI risk people in the know are afraid of is not a flood of AI-generated articles, but something that probably can't be achieved yet with current levels of AI (and more slop generated from current-day AI won't hasten the advent of). On the other hand, modern-day greenhouse gases are exactly the greenhouse gases that climate activists are afraid of in the limit.

The AI risks people in the know are afraid of, are indeed what I listed: pretty much any person, anywhere, using an existing model running off their laptop to foment real-world violence or spread disinformation en masse on social media, or use it on a cheap swarm of weaponized/suicide drones, for a few examples

That's what we're already seeing today, so we know the risk is there and has a probability of 100%. The "skynet" AI risk is far more fringe and farfetched.

So, like you said about climate change, the harm can come from 1 person. In the case of climate change, though, the risks people in the know are afraid of, aren't "some guy willing to spin a stick for long enough to start a fire, or to feed and protect a flock of cows"

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#720

Post by 3rd3 »

dragonwriter wrote: post #575
JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

Yudkowsky’s call for declaring a global AI halt and engaging in unlimited warfare with anyone who defies it poses a greater and more immediate existential risk to humanity than AI ever could; it is largely mitigated (though still a bigger X-Risk than AI) because no one with the capacity to implement it in any substantive way takes him seriously at the moment. But the people selling AI X-Risk that anyone with any poli…

> Yudkowsky’s call for declaring a global AI halt and engaging in unlimited warfare with anyone who defies it

Could you provide a link to where he said this?

Post reply on HN