Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pausing AI Developments Isn't Enough. We Need to Shut It All Down

time.com

Post reply on HN
Print view View original

41–50 of 414 posts

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#41

Post by Tuna-Fish »

ilaksh wrote: post #34
dougmwne wrote: post #14

The Butlerian Jihad is officially no longer science fiction. Man may not be replaced. “The Letter” was obviously self serving drivel from people who want time to get in the game. Google does care about AI existential risk, they care about beating Microsoft by any means possible, including declaring a moratorium, but continuing to make progress behind the scenes. This guy is the real deal. I can imagine he would perso…

What does the word "super" mean to you? In some ways, GPT4 is already superintelligent. So is ChatGPT 3.5. Do you know anyone who can translate natural language to code as fast as GPT 3.5, or, given a few paragraphs of reference text, perfectly tutor a child on any subject under the sun at a moment's notice like GPT4 is doing for Khan Academy? How many artists do you know who can produce almost any style of artwork w…

Being superhuman at producing art is not existentially threatening. Being superhuman about convincing people or writing code would be.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#42

Post by Imnimo »

Analemma_ wrote: post #40
Imnimo wrote: post #35

I feel like if I actually believed that the death of all biological life was imminent and that the only way to forestall it was to bomb data centers, I'd spend a lot more time planning how to bomb data centers and a lot less time curating Shoggoth memes on my web forum. On the other hand, if I were more concerned with self-aggrandizement, writing doomsaying thinkpieces for Time is exactly what I'd spend my time on.

These types of arguments-- "if you actually believed X, you'd be doing Y"-- always ring false to me. If you genuinely believed that the death of all biological life was imminent and the only way to forestall it was to bomb data centers and you were a normal, functional human being who did not want to bomb data centers, you'd probably sink into hopeless depression. Which is, in fact, exactly what Eliezer has done - if…

I think if I seriously thought all biological life was about to be destroyed, I could get over my distaste for bombing data centers.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#44

Post by karp773 »

Tuna-Fish wrote: post #41
ilaksh wrote: post #34

Earlier quoted context omitted.

What does the word "super" mean to you? In some ways, GPT4 is already superintelligent. So is ChatGPT 3.5. Do you know anyone who can translate natural language to code as fast as GPT 3.5, or, given a few paragraphs of reference text, perfectly tutor a child on any subject under the sun at a moment's notice like GPT4 is doing for Khan Academy? How many artists do you know who can produce almost any style of artwork w…

Being superhuman at producing art is not existentially threatening. Being superhuman about convincing people or writing code would be.

Wait until AI learns to generate propaganda memes and share them on social media.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#45

Post by Tuna-Fish »

brucethemoose2 wrote: post #18
oifjsidjf wrote: post #9

The cat is out of the box. You really think eg China will also pause this? Myopic thinking: the country that will have the most powerful AI first will be the leader in everything. >> If the policy starts with the U.S., then China needs to see that the U.S. is not seeking an advantage but rather trying to prevent a horrifically dangerous technology which can have no true owner and which will kill everyone in the U.S.…

I dunno, the end goal (a pseudo AGI) is more like a nuclear weapon than the advent of electricity. The concern is real, even if we are farther away than the author realizes and the risk (like the risk if nuclear war) is not totally insurmountable even if it is extremely difficult. The first nuclear bombs couldn't end life on Earth either, but it wasn't long before they could, and the scientists working on them saw th…

OT, but it's a pet peeve:

> but it wasn't long before they could

The total amount of nuclear weapons ever built is laughably inadequate for the task of ending life on earth. They would not end human life either, and even ending human civilization (as in, agriculture and organized society) is off by many orders of magnitude.

Nuclear war would be horrible, but the actual impact got massively overinflated, largely because of good reasons.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#46

Post by Analemma_ »

Imnimo wrote: post #42
Analemma_ wrote: post #40

Earlier quoted context omitted.

These types of arguments-- "if you actually believed X, you'd be doing Y"-- always ring false to me. If you genuinely believed that the death of all biological life was imminent and the only way to forestall it was to bomb data centers and you were a normal, functional human being who did not want to bomb data centers, you'd probably sink into hopeless depression. Which is, in fact, exactly what Eliezer has done - if…

I think if I seriously thought all biological life was about to be destroyed, I could get over my distaste for bombing data centers.

No, that doesn't follow. It's possible to think that all biological life was about to be destroyed but also believe some combination of

1) it's hopeless to bomb data centers because I'd quickly be stopped by the authorities and remaining data centers would beef up security, and nothing would be accomplished,

2) in the past, other people who have had beliefs of this sort ("violence is the only answer") have been wrong and I should not act on them even if I am sure "this time is different", because those other people also thought this time was different, and

3) just having a deeply ethically-ingrained prohibition against violence like this which is not easy to overcome through intellectual rationalization alone.

These are not mutually exclusive, and if you did believe all of them at once, I think it's reasonable to assume you'd be in a serious depressive spiral.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#47

Post by Imnimo »

Analemma_ wrote: post #46
Imnimo wrote: post #42

Earlier quoted context omitted.

I think if I seriously thought all biological life was about to be destroyed, I could get over my distaste for bombing data centers.

No, that doesn't follow. It's possible to think that all biological life was about to be destroyed but also believe some combination of 1) it's hopeless to bomb data centers because I'd quickly be stopped by the authorities and remaining data centers would beef up security, and nothing would be accomplished, 2) in the past, other people who have had beliefs of this sort ("violence is the only answer") have been wrong…

Israel believes that Iran having a nuclear weapon is an existential threat. What do they do?

https://en.wikipedia.org/wiki/Assassination_of_Iranian_nucle... https://en.wikipedia.org/wiki/2020_Iran_explosions

I find it hard to believe that Yudkowsky et al. think we're facing a threat that is many times greater in magnitude, and yet are completely unwilling to act.

Here's what he advocates for in the article:

>If intelligence says that a country outside the agreement is building a GPU cluster, be less scared of a shooting conflict between nations than of the moratorium being violated; be willing to destroy a rogue datacenter by airstrike.

>Make it explicit in international diplomacy that preventing AI extinction scenarios is considered a priority above preventing a full nuclear exchange, and that allied nuclear countries are willing to run some risk of nuclear exchange if that’s what it takes to reduce the risk of large AI training runs.

Does this sound like someone who has a deeply ethically-ingrained prohibition against violence?

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#48

Post by myrryr »

13years wrote: post #23
myrryr wrote: post #22

Earlier quoted context omitted.

I think without AGI we are dead anyway, AGI gives us a roll at the dice to survive as a species. We can't keep going in the direction we have been and end up somewhere survivable.

AGI is a continuation of what we have been doing. Technological advancements at a pace that surpasses our ability to reason about their effects on us and the world. I see two potential outcomes as most likely. We have control of power that we are not able to responsibly manage, or we are managed by power we can not control.

Totally agree but "have control of power that we are not able to responsibly manage", is becoming increasingly untenable.

"we are managed by power we can not control" sounds like a step up from here, since we corrupt the things we can control.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#49

Post by brucethemoose2 »

Tuna-Fish wrote: post #45
brucethemoose2 wrote: post #18

Earlier quoted context omitted.

I dunno, the end goal (a pseudo AGI) is more like a nuclear weapon than the advent of electricity. The concern is real, even if we are farther away than the author realizes and the risk (like the risk if nuclear war) is not totally insurmountable even if it is extremely difficult. The first nuclear bombs couldn't end life on Earth either, but it wasn't long before they could, and the scientists working on them saw th…

OT, but it's a pet peeve: > but it wasn't long before they could The total amount of nuclear weapons ever built is laughably inadequate for the task of ending life on earth. They would not end human life either, and even ending human civilization (as in, agriculture and organized society) is off by many orders of magnitude. Nuclear war would be horrible, but the actual impact got massively overinflated, largely becau…

The initial fission stage is the "hard" part of a nuke, but the subsequent fusion stages are much easier to scale up. Arbitrarily large nuclear bombs are theoretically "easy" to make by chaining increasingly large fusion stages one after the other, as they aren't limited by expensive fissile material.

https://blog.nuclearsecrecy.com/2012/09/12/in-search-of-a-bi...

Hence scientists extrapolated nukes really could end life on Earth, and in a slightly different reality than ours, they may have.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#50

Post by jamilton »

Imnimo wrote: post #47
Analemma_ wrote: post #46

Earlier quoted context omitted.

No, that doesn't follow. It's possible to think that all biological life was about to be destroyed but also believe some combination of 1) it's hopeless to bomb data centers because I'd quickly be stopped by the authorities and remaining data centers would beef up security, and nothing would be accomplished, 2) in the past, other people who have had beliefs of this sort ("violence is the only answer") have been wrong…

Israel believes that Iran having a nuclear weapon is an existential threat. What do they do? https://en.wikipedia.org/wiki/Assassination_of_Iranian_nucle... https://en.wikipedia.org/wiki/2020_Iran_explosions I find it hard to believe that Yudkowsky et al. think we're facing a threat that is many times greater in magnitude, and yet are completely unwilling to act. Here's what he advocates for in the article: >If intel…

The government of Israel is not particularly known for being ethical.
Post reply on HN