Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

Post by dnissley » Thu, Nov 02, 2023, 10:10 AM UTC

lewhoo wrote: post #273

RandomLensman wrote: post #240

Earlier quoted context omitted.

Unlike for AI, in those fields we do not need to stack endless hypotheses on top of each other to arrive at something bad (and even then some prohibitions are tenuous in their reasoning at best).

I don't know about that. Cloning/modifying humans has potentially so much benefit and yet we seem to stack hypotheses on top of it and conclude it's not worth it. There is also a lot to be said about all the hypotheses of killing all the Hitlers preemptively. I think that whatever we do, we usually arrive at a bit modified trolley problem and try to go from there. We really should never expect that the one particular…

Post by TeMPOraL » Thu, Nov 02, 2023, 10:17 AM UTC

jeffparsons wrote: post #229

YetAnotherNick wrote: post #222

Earlier quoted context omitted.

Global coordination using coercion has worked for nukes. It's very hard but not impossible to ban NVidia or any competitor and create spy network in all large companies and if any engineer is found to create AI, mark them as terrorist and give them POW treatment. And monitor chip fab like nuclear enrichment facility and launch war with country found to create chip fab, which I belive is hard to do it in complete secr…

Sure, I think that kind of "coordination" is a little more credible than what I thought Eliezer was proposing. Maybe I just didn't read carefully enough. One big difference here, however, is that the barrier to entry is lower. You can start doing meaningful AI R&D on existing hardware with _one_ clever person. The same was never true for nuclear weapons, so the pool of even potential players was pretty shallow for a…

Post by lewhoo » Thu, Nov 02, 2023, 10:21 AM UTC

dnissley wrote: post #291

lewhoo wrote: post #273

Earlier quoted context omitted.

I don't know about that. Cloning/modifying humans has potentially so much benefit and yet we seem to stack hypotheses on top of it and conclude it's not worth it. There is also a lot to be said about all the hypotheses of killing all the Hitlers preemptively. I think that whatever we do, we usually arrive at a bit modified trolley problem and try to go from there. We really should never expect that the one particular…

The best argument against human cloning/gmo is simply that we can't do this well on other animal species just yet, so probably we should be hold off on the humans. It's hyper pragmatic from an ethics perspective, not a trolley problem or stack of hypotheses at all.

Post by shafyy » Thu, Nov 02, 2023, 10:24 AM UTC

toth wrote: post #276

zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

Post by dnissley » Thu, Nov 02, 2023, 10:29 AM UTC

upupupandaway wrote: post #185

FeepingCreature wrote: post #154

Earlier quoted context omitted.

The aim is to stop it from becoming easy. The drug war would look very different if good cocaine was only produced in one country with extremely costly chemistry machines from one company.

If good cocaine was only produced in one country with extremely costly machines, a good-enough alternative would appear almost immediately. This has happened millions of times in the history of humanity, I refuse to believe that people haven't learned this lesson.