Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

331–340 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#331

Post by espadrine »

homefree wrote: post #87
brutusborn wrote: post #83

Earlier quoted context omitted.

Who do you find persuasive and what material should I read/watch to understand their POV? So far I’ve read a bunch of lesswrong posts and listened to some Eliezer talks but still can’t understand the basis of their arguments or when I do, they seem very vague. The only suggestion that makes sense to me is from the FEP crowd. Essentially if someone sets up at AI with an autopoietic mechanism then it would be able to t…

There’s a decent podcast interview between Sam Harris and Eliezer Yudkowsky (on Sam’s pod), I think that’s a decent introduction and they break down the ideas in a way that’s more approachable for someone curious about it. For my personal quick summary I have earlier comments: https://news.ycombinator.com/item?id=36104090

Thanks for summarizing this argument. You can adapt it with many other entities whose risk is in practice managed. Replace the entity with “super-intelligent humans,” “extra-terrestrials,” “coronaviruses with higher incubation and mortality,” “omnivorous ant swarms”…

1. are possible.

2. Unaligned are an existential risk likely to wipe out humanity.

3. We have no idea how to align .

What those example entities have that AGI doesn’t, is self-reproduction, a significant and hard (in the sense of Moravec’s paradox) achievement for a species to have, yet one that significantly increases its survival probability.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#332

Post by afarviral »

reducesuffering wrote: post #252
persnickety wrote: post #149

> Every new technology is developed and deployed the same way: > You make a prototype, try it at a small scale, make limited deployment, fix the problems, make it safer, and then deploy it more widely. This makes an assumption: that the problematic technology is an intentional development rather than an emergent feature of an intentionally developed technology. We already have a name for such emergent features: "bugs…

Yep. And there are already 20 year old viruses out there, autonomously following their programming, causing issues aligned with no human, as the original humans do not profit off it anymore, and unable to “be turned off”…

Really? What are some examples?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#333

Post by jeffparsons »

TeMPOraL wrote: post #293
jeffparsons wrote: post #229

Earlier quoted context omitted.

Sure, I think that kind of "coordination" is a little more credible than what I thought Eliezer was proposing. Maybe I just didn't read carefully enough. One big difference here, however, is that the barrier to entry is lower. You can start doing meaningful AI R&D on existing hardware with _one_ clever person. The same was never true for nuclear weapons, so the pool of even potential players was pretty shallow for a…

> Maybe I just didn't read carefully enough. It's this. This kind of coordination is exactly what he's been proposing ever since the current AI safety discussion started last year. However, he isn't high-profile enough to be reported on frequently, so people only remember and quote the "bomb data centres" example he used to highlight what that kind of coordination means in real terms.

Ah, okay. Thanks. The world makes a little more sense again!

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#334

Post by tumult »

api wrote: post #330
jefftk wrote: post #325

Earlier quoted context omitted.

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

So ask it how to stop someone from doing this. Ask it how to augment the human immune system. Train AIs on the structures of antibiotics and antiviral drugs and have them predict novel ones. Create an AI that can design vaccines against novel agents. Those things could stop our terrorist and could also stop the next breakout zoonotic virus or “oops!” in a lab somewhere. Intelligence amplification works for everyone.…

You've also made an assumption. That attack vs. react is is symmetrical. It's not.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#335

Post by pjc50 »

duckmysick wrote: post #201
timmytokyo wrote: post #187

Earlier quoted context omitted.

His solution -- assuming it's the same as the one proposed by Yudkowsky, who he quotes -- is launching air strikes against data centers hosting AI computation. So yeah, imaginary.

I wonder what would happen if the AI computation is co-hosted in data centers with civilian data.

Yukowsky is presumably a "we have to kill the humans in order to save them" maximalist.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#336

Post by pjc50 »

scarmig wrote: post #197
timmytokyo wrote: post #187

Earlier quoted context omitted.

His solution -- assuming it's the same as the one proposed by Yudkowsky, who he quotes -- is launching air strikes against data centers hosting AI computation. So yeah, imaginary.

Ironically, it seems the only way to stop someone from building an omnipotent AI that could destroy the world is to build an omnipotent AI under your control faster than any of the people you need to destroy. If you believe in that sort of thing.

This is just the plot of War Games (1983).

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#337

Post by toth »

M95D wrote: post #315
toth wrote: post #276

Earlier quoted context omitted.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

> He's pretty sure that human civilization will be extinct this century. > There is no scenario where climate change leads to human extinction. He was talking about the human CIVILIZATION...

You are right, but "human civilization will be extinct" is an awkward phrase.

Human extinction via climate change is off the table.

The end of human civilization? Depends on what one means by it. If it means we'll go back to being hunter gatherers it's ridiculous. Some civilizational wide regression is not impossible, but even the direst IUPAC scenario projects only a one digit drop in global GDP (of course, that's the average, it will be much worse in some regions).

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#338

Post by Mordisquitos »

RandomLensman wrote: post #139
troupo wrote: post #123

Earlier quoted context omitted.

> The laws we have already cover malevolent and abusive uses of software applications. They don't. Or not nearly enough. Otherwise you wouldn't have automated racial profiling, en masse face recognition, credit and social scoring etc. And it's going to get worse. Because AI is infallible, right? Right?! That's why, among other thing, EU AI regulation has the following: - Foundation models have to be thoroughly docume…

AI not being allowed in high-risk applications is a bad idea, same for explicable in human terms (cannot do that now for human decisions anyway). The latter would kind of rule out any quantum computing, too, as the only real explanation there is through maths, not words.

> The latter would kind of rule out any quantum computing, too, as the only real explanation there is through maths, not words.

Maths are human terms.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#339

Post by jacquesm »

api wrote: post #330
jefftk wrote: post #325

Earlier quoted context omitted.

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

So ask it how to stop someone from doing this. Ask it how to augment the human immune system. Train AIs on the structures of antibiotics and antiviral drugs and have them predict novel ones. Create an AI that can design vaccines against novel agents. Those things could stop our terrorist and could also stop the next breakout zoonotic virus or “oops!” in a lab somewhere. Intelligence amplification works for everyone.…

There is no symmetry between destruction and creation/prevention of destruction, destruction is far easier.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#340

Post by pjc50 »

timmytokyo wrote: post #187
topspin wrote: post #142

Earlier quoted context omitted.

> "Stop" is not regulatory capture. "Stop" is a fiction that exists exclusively inside your head. Unless your solution anticipates a planet wide GULAG system run by a global government your "stop" is infeasible.

His solution -- assuming it's the same as the one proposed by Yudkowsky, who he quotes -- is launching air strikes against data centers hosting AI computation. So yeah, imaginary.

Ah, so we're going to start the global nuclear exchange with China based on the paranoid fantasies of some guy on Twitter? And this is supposed to be taken seriously?
Post reply on HN