Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pausing AI Developments Isn't Enough. We Need to Shut It All Down

time.com

Post reply on HN
Print view View original

21–30 of 414 posts

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#21

Post by alwayslikethis »

13years wrote: post #10
gremlinsinc wrote: post #6

I'm anxious, excited, and nervous about AI, but this feels more akin to luddite-thinking, the world's about to shift, but I'm hoping the shift is a positive one.

I agree to the point that it is totally unrealistic that it shuts down. There are no difficult barriers for entry into this space. No possible way of containment other than imposing hardware restrictions. However, it is indeed a shift, and a massive power shift. With any substantial increase of power comes abuse of that power. The more power we have there is a tendency for humanity to manage it unwisely. I've covered…

Hardware restrictions are unrealistic given all the GPUs that already exist. It's not like centrifuges which are big and few in number. RTX 4090s are everywhere.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#22

Post by myrryr »

13years wrote: post #11
oifjsidjf wrote: post #9

The cat is out of the box. You really think eg China will also pause this? Myopic thinking: the country that will have the most powerful AI first will be the leader in everything. >> If the policy starts with the U.S., then China needs to see that the U.S. is not seeking an advantage but rather trying to prevent a horrifically dangerous technology which can have no true owner and which will kill everyone in the U.S.…

> Imagine being so naive you belive this could ever happen. Agreed, not a rational scenario > Also: imagine the year is 1900. You are saying that steam power and electricity is causing way too many changes way too fast so they put a moratorium on it until the year 2500. However, the risk scenarios are indeed real. We have quite a dilemma on our hands.

I think without AGI we are dead anyway, AGI gives us a roll at the dice to survive as a species.

We can't keep going in the direction we have been and end up somewhere survivable.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#23

Post by 13years »

myrryr wrote: post #22
13years wrote: post #11

Earlier quoted context omitted.

> Imagine being so naive you belive this could ever happen. Agreed, not a rational scenario > Also: imagine the year is 1900. You are saying that steam power and electricity is causing way too many changes way too fast so they put a moratorium on it until the year 2500. However, the risk scenarios are indeed real. We have quite a dilemma on our hands.

I think without AGI we are dead anyway, AGI gives us a roll at the dice to survive as a species. We can't keep going in the direction we have been and end up somewhere survivable.

AGI is a continuation of what we have been doing. Technological advancements at a pace that surpasses our ability to reason about their effects on us and the world.

I see two potential outcomes as most likely. We have control of power that we are not able to responsibly manage, or we are managed by power we can not control.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#24

Post by 13years »

alwayslikethis wrote: post #21
13years wrote: post #10

Earlier quoted context omitted.

I agree to the point that it is totally unrealistic that it shuts down. There are no difficult barriers for entry into this space. No possible way of containment other than imposing hardware restrictions. However, it is indeed a shift, and a massive power shift. With any substantial increase of power comes abuse of that power. The more power we have there is a tendency for humanity to manage it unwisely. I've covered…

Hardware restrictions are unrealistic given all the GPUs that already exist. It's not like centrifuges which are big and few in number. RTX 4090s are everywhere.

Depends on what is required for AGI/ASI. Some say we still need much more power, others think we are almost there. So, yes, unrealistic if we are almost there already.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#26

Post by anongraddebt »

scarmig wrote: post #19
oifjsidjf wrote: post #8

Earlier quoted context omitted.

This is the one moment he has been waiting his entire life for. He literally waited his entire life for this moment, obviously he's gonna milk it to every drop.

Self-aggrandizement isn't anything foreign to Eliezer, but he's genuinely terrified. Milking it for every drop doesn't really capture what he's doing.

Yeah, I wonder sometimes how many people gloss over what it would mean for an “ASI” to exist. We would - if successful - effectively be creating a supernatural being (at least, there wouldn’t be much of a distinction between it and one).

Sure, there’s hype, and FUD, and fatalism, etc. But, if you believe the creation of such a thing is within your lifetime then it would be difficult to find many higher priority issues to prepare for/help solve/vote on/etc.

In reality, we all likely still downplay the risk by assessing the limit on the downside as a relatively quick extinction of life on earth. There are many things one can imagine might be a lot worse than death.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#27

Post by dspeyer »

mattrighetti wrote: post #25

> We are not ready. We are not on track to be significantly readier in the foreseeable future. How do you know if you're ready for something like this? Legitimate question

Off the top of my head...

    * A plan exists
    * The plan itself predicts non-obvious results in smaller systems, and those tests have passed (prediction written *before* running test)
    * A bunch of smart people have looked at it and said, "Yes, this looks plausible"
The closest thing to a plan is RLHF, which has failed every toy problem its been thrown at and made everyone in the field say "even if this worked in toy problems, it wouldn't generalize".

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#28

Post by sullyj3 »

oifjsidjf wrote: post #9

The cat is out of the box. You really think eg China will also pause this? Myopic thinking: the country that will have the most powerful AI first will be the leader in everything. >> If the policy starts with the U.S., then China needs to see that the U.S. is not seeking an advantage but rather trying to prevent a horrifically dangerous technology which can have no true owner and which will kill everyone in the U.S.…

I'm sure he agrees the chance is remote, which is why he's so afraid. He's articulating what would be required, he literally uses the word "miracle" in the article.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#29

Post by jamestimmins »

I've used GPT-4, and while extremely impressive, it doesn't feel like we're all that much closer to super-intelligent AI than we were last month or last year. It feels like Google on steroids, but the gap between GPT-4 and AGI still feels massive. This seems like so much putting the cart way ahead of the horse.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#30

Post by [deleted] »

anongraddebt wrote: post #26
scarmig wrote: post #19

Earlier quoted context omitted.

Self-aggrandizement isn't anything foreign to Eliezer, but he's genuinely terrified. Milking it for every drop doesn't really capture what he's doing.

Yeah, I wonder sometimes how many people gloss over what it would mean for an “ASI” to exist. We would - if successful - effectively be creating a supernatural being (at least, there wouldn’t be much of a distinction between it and one). Sure, there’s hype, and FUD, and fatalism, etc. But, if you believe the creation of such a thing is within your lifetime then it would be difficult to find many higher priority issue…

[deleted]
Post reply on HN